Company directory
Every establishment in the index, grouped by NAICS sector, with Dewey classes.
/xml/directory.xml
Public XML subroutine
Crosswalk publishes the crawled companies as XML — a NAICS-grouped directory, a sitemap of source URLs, an RSS feed, and OpenSearch. CORS is open, so other catalogs can read every establishment without a key.
Every establishment in the index, grouped by NAICS sector, with Dewey classes.
/xml/directory.xml
Source URLs and catalog pages for search engines and other crawlers.
/xml/sitemap.xml
Recently classified pages, tagged with NAICS and Dewey categories.
/xml/feed.xml
Lets browsers and other engines add Crosswalk as a search provider.
/xml/opensearch.xml
Download the seed crawl plus any pages classified in this browser. The public feeds always serve the shared crawled index.
<?xml version="1.0" encoding="UTF-8"?>
<directory xmlns="urn:crosswalk:directory:v1" generated="preview" origin="" pages="4">
<meta>
<title>Crosswalk public company directory</title>
<description>Establishments and sources crawled from the open web, each assigned a NAICS industry code and a Dewey Decimal class. CORS is open so other indexes can read this feed.</description>
</meta>
<sector code="54" label="Professional, Scientific, and Technical Services" count="4">
<establishment id="github-react">
<name>facebook/react: The library for web and native user interfaces</name>
<url>https://github.com/facebook/react</url>
<catalog>/page/github-react</catalog>
<domain>github.com</domain>
<snippet>React is a JavaScript library for building user interfaces. Declarative, component-based, and used by Meta, Netflix, and thousands of production sites.</snippet>
<dewey code="005" confidence="0.93">Computer programming, programs & data</dewey>
<naics code="5415" confidence="0.9">Computer Systems Design and Related</naics>
<keywords>
<keyword>javascript</keyword>
<keyword>ui</keyword>
<keyword>open source</keyword>
<keyword>frontend</keyword>
<keyword>library</keyword>
</keywords>
<crawled>2026-08-12T09:00:00Z</crawled>
</establishment>
<establishment id="arxiv-transformer">
<name>Attention Is All You Need</name>
<url>https://arxiv.org/abs/1706.03762</url>
<catalog>/page/arxiv-transformer</catalog>
<domain>arxiv.org</domain>
<snippet>We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.</snippet>
<dewey code="006" confidence="0.94">Special computer methods</dewey>
<naics code="5417" confidence="0.88">Scientific Research and Development</naics>
<keywords>
<keyword>transformer</keyword>
<keyword>attention</keyword>
<keyword>neural networks</keyword>
<keyword>machine learning</keyword>
<keyword>nlp</keyword>
</keywords>
<crawled>2026-08-12T09:00:00Z</crawled>
</establishment>
<establishment id="nist-cyber">
<name>NIST Cybersecurity Framework</name>
<url>https://www.nist.gov/cyberframework</url>
<catalog>/page/nist-cyber</catalog>
<domain>nist.gov</domain>
<snippet>The NIST Cybersecurity Framework provides guidance for organizations to manage and reduce cybersecurity risk, organized around Identify, Protect, Detect, Respond, and Recover.</snippet>
<dewey code="005" confidence="0.9">Computer programming, programs & data</dewey>
<naics code="5416" confidence="0.82">Management, Scientific, and Technical Consulting</naics>
<keywords>
<keyword>cybersecurity</keyword>
<keyword>risk</keyword>
<keyword>framework</keyword>
<keyword>standards</keyword>
<keyword>information security</keyword>
</keywords>
<crawled>2026-08-12T09:00:00Z</crawled>
</establishment>
<establishment id="w3-html">
<name>HTML 5.2 — W3C Recommendation</name>
<url>https://www.w3.org/TR/html52/</url>
<catalog>/page/w3-html</catalog>
<domain>w3.org</domain>
<snippet>This specification defines the 5th major version, second minor revision of the core language of the World Wide Web: the Hypertext Markup Language (HTML).</snippet>
<dewey code="006" confidence="0.86">Special computer methods</dewey>
<naics code="5415" confidence="0.78">Computer Systems Design and Related</naics>
<keywords>
<keyword>html</keyword>
<keyword>web standards</keyword>
<keyword>markup</keyword>
<keyword>browser</keyword>
</keywords>
<crawled>2026-08-12T09:00:00Z</crawled>
</establishment>
</sector>
</directory>
Each sector has its own XML document. Other indexes can subscribe per industry.
Robots.txt points crawlers at the sitemap. Alternate feed links are in the document head.