For the html5 mode to work properly, you must configure your webserver so that requests to directories that don't exist get rewritten to index.html.
Google does not properly follow links in angularjs apps yet, therefore you must create a sitemap for all of your routes. This sounds like a pain to do, however, with proper build processes this can be a very automated process. (gulp, grunt, etc.)
One commonly missed problem with indexing angular apps is things like pagination and content that shows up after clicking a button. If these actions do not change the url, google will not crawl it. For example, say you have a page with a table using pagination and it has 3 pages. Google will only crawl the first page unless each page has a different url route such as /table/page/1/table/page/2/table/page/3