What happens if the option contract is not squared off on the expiry date?

Index options are cash-settled, and the implications are as follows:

If the index options are bought:
Contracts expiring ITM - Securities Transaction Tax(STT) is charged on exercised contracts at the rate of 0.125% of intrinsic value (how much in-the-money the option is), i.e. intrinsic value * quantity and not on the total contract value. Brokerage will be charged on both sides, i.e. when the options are bought and when they are settled on the expiry day.

Contracts expiring OTM - OTM option contracts expire worthlessly. The entire amount paid as a premium will be lost. Brokerage will only be charged on one side, which is when the options are purchased, and not when they expire worthless on the expiry day.

If the index options are shorted or sold:

STT is charged only for the sell-side, which means the STT is charged when initiating the short. So, there will be no STT impact on expiry. Depending on the moneyness of the option contract, the trader gets to keep the premiums received.