Many astrophysical and galaxy-scale cosmological problems require a well-determined gravitational potential. Globular clusters (GCs) surrounding galaxies can be used as dynamical tracers of the total and dark matter distribution at large (kpc) scales. A useful tool which has been used to constrain the gravitational potential of the Milky Way (MW) are the action-angles of resolved stars in stellar streams. Actions are integrals of motion with an intuitive physical meaning - e.g. the radial action quantifies the radial oscillation of the orbit - and therefore make excellent orbit labels. A problem in external galaxies is that individual stars cannot be resolved, however GCs can similarly trace accretion events. I will present a method, “adaptive dynamics”, showing how we can use actions of accreted GCs to constrain the gravitational potential. The first applications of this method are carried out in the MW-like magneto hydrodynamical Auriga simulations. I will show how sensitive this method is for the Galactic GCs, for which we have full 6D phase space information by combining Gaia DR2 and literature values, and how we can extend it to external GCs, where we have less information.