To manage a group, you need to have at least level 3 (Admin) access in that group. More on Access Level, please read "What are the access levels in Signals platform?".

To manage your group, simply click on the group name from the Group Management Interface ("Where can I see all my groups?"). Then click on any group that you want to edit. 

A "Edit User Group" interface will pop up, shown below. You can add, remove, and edit group members access level in this interface.