Variety of situations affect who can access a task:
Task in a private project
Task followers and project members
Task in a public project
Task followers, project members, and group members
Those who have access to the parent task
When a task is no shared in any project and only a few followers are added to the task, only those followers would be able to access the task.