Google PageSpeed is a useful tool that gauges the speed of your site, but it shouldn't be taken as gospel. First of all, it doesn't look at your website as a whole and it doesn't take certain things under consideration. It really just looks at your site's code, compares it to a checklist, and then spits out a number. Since it just reads the HTML on your page, you can sometimes get confusing results like when one site loads in 2 seconds but gets a 60, and then another site loads in 8 seconds but gets in the 80s.
Digging a little deeper, what they "want" you to do is to minify the resources and deliver them inline. That's all fine and good for a static, single page site with a few files, but god help you if the site is large, complex, old, or a combination of the three. Good luck maintaining it and not breaking it or having bugs. If everything else is optimized and the page fully renders in a couple seconds, then you shouldn't worry about getting docked a few points here.
Another incredibly common issue that gets you points taken off is using Google Analytics and Google fonts. Yes, you read that correctly, Google PageSpeed penalizes you for using Google Analytics and Google fonts. That would be hilarious if it wasn't such a pain. Although, something to consider is one of Google Fonts' selling points - caching. Sure, it's very heavy, but it's so ubiquitous that you can be fairly certain the majority of your site's visitors will already have them cached. So, once again, don't worry about that.
Regarding testing your site, real accuracy is gaining insight from multiple sources. It's true for reading the news and getting a second opinion from a doctor; it's true for your page speed as well. In addition to PageSpeed Insights, you can use Pingdom and GTMetrix. Another great tool is Google Lighthouse, which gives you way more information. You can see what your site scores in accessibility, SEO, best practices, performance, and PWA.
Even google.com has a 92/100 for desktop and 62/100 for mobile. Youtube has a 70 and 51. Facebook has a 79 and 48. Obviously these sites are probably not similar to yours, but if your boss or client is demanding scores of 100 across the board, just show them this article.
Or you can build your site with React using Gatsby; this site was built with Gatsby and scores 100 in every single category of the Lighthouse Audit.
You can read my full guide on how to build a full site using Gatsby in this article.