Filling in the Organization field in the support portal is necessary to associate you with an account that holds a valid support contract.
Notifications
All support users receive notifications about cases related to their own organization:
They are notified each time a ticket has been created.
They are notified when the organization has been added to an existing ticket.
They are able to 'opt in' to see updates about each ticket. If they do not opt in, they do not get updates on the case.
How are people removed?
If there are others within your organization that were previously copied on tickets who no longer need access to your support tickets, ask Support to remove them from the organization.