Uninstall just vagrant
This will remove just the vagrant package itself.
sudo apt-get remove vagrant
Uninstall vagrant and its dependencies
To remove the vagrant package and any other dependant package which are no longer needed from Ubuntu.
sudo apt-get remove --auto-remove vagrant
Purging vagrant (your config/data too)
If you also want to delete your local/config files for vagrant then this will work.
sudo apt-get purge vagrant
To delete configuration and/or data files of vagrant and it's dependencies from Ubuntu then execute.
sudo apt-get purge --auto-remove vagrant