Navigate to Billy's admin panel in your Slack workspace (in the sidebar, click on the app name at the bottom of the screen). Click on 'About' tab then on 'Configuration' button. You'll find a section called 'Bot User', you can click on 'Edit' in this section. You should now see a modal called 'Edit bot name'.