The Links Analyzer integrates with third party services and web crawlers such as Bing to retrieve generic stats about the indexing status of a certain link. The fact that the Links Analyzer reports some links as not indexed doesn't mean that there is an issue with those links or that links are not indexed by Google, indeed this stat is totally unrelated to Google indexing.

If your links are reported as a red icon 'Not indexed link' or a yellow icon 'Informations not available' it doesn't mean that Google has not indexed your links. It could be the case that there are not enough informations stored on web crawlers or that web crawlers have not indexed your links, but Google has actually indexed them all.
To check if Google has really indexed or not indexed a certain link you have to search for that link in the Google search using 'site:' as a prefix of your link, for example: 'site:http://www.yoursite.com/yourlink.html'
So assuming that you want to know if Google has indexed the link 'https://storejextensions.org/extensions/jsitemap_professional.html' you must perform the following search and if the link is indexed Google will show you exactly 1 result:

You can just ignore the stats of the Links Analyzer indexing status once you have checked that Google has actually indexed your links. The Links Analyzer can be also configured to use Web Crawler or Bing for indexing stats and to use different URLs criteria, playing with these settings may provide you more exact stats retrieval in certain cases. If you are using the Web Crawler engine and all your links have no informations about the indexed status, you can try to enter either the value 'web-google__url' or 'web-bing__url'.

If instead you find that Google has really not indexed some of your links, ensure that they are included in the sitemap and that the sitemap has been correctly submitted to Google.
As a further step you can use the feature of Google Webmasters Tools named 'Fetch as Google' that forces Google to retrieve and index specific links of your website.

Notice that it's up to Google to choose if your links should be indexed or not, so there could be cases in which some of your URLs are simply discarded or ignored by Google for several reasons, most common ones are content duplication, canonicalization and lack of content.