Index 102 pages0 crawled here

Public XML subroutine

Open the index to the rest of the internet.

Crosswalk publishes the crawled companies as XML — a NAICS-grouped directory, a sitemap of source URLs, an RSS feed, and OpenSearch. CORS is open, so other catalogs can read every establishment without a key.

Company directory

Every establishment in the index, grouped by NAICS sector, with Dewey classes.

/xml/directory.xml

Open XML

Sitemap

Source URLs and catalog pages for search engines and other crawlers.

/xml/sitemap.xml

Open XML

RSS feed

Recently classified pages, tagged with NAICS and Dewey categories.

/xml/feed.xml

Open XML

OpenSearch

Lets browsers and other engines add Crosswalk as a search provider.

/xml/opensearch.xml

Open XML

This session as XML

Download the seed crawl plus any pages classified in this browser. The public feeds always serve the shared crawled index.

<?xml version="1.0" encoding="UTF-8"?>
<directory xmlns="urn:crosswalk:directory:v1" generated="preview" origin="" pages="4">
  <meta>
    <title>Crosswalk public company directory</title>
    <description>Establishments and sources crawled from the open web, each assigned a NAICS industry code and a Dewey Decimal class. CORS is open so other indexes can read this feed.</description>
  </meta>
  <sector code="54" label="Professional, Scientific, and Technical Services" count="4">
    <establishment id="github-react">
      <name>facebook/react: The library for web and native user interfaces</name>
      <url>https://github.com/facebook/react</url>
      <catalog>/page/github-react</catalog>
      <domain>github.com</domain>
      <snippet>React is a JavaScript library for building user interfaces. Declarative, component-based, and used by Meta, Netflix, and thousands of production sites.</snippet>
      <dewey code="005" confidence="0.93">Computer programming, programs &amp; data</dewey>
      <naics code="5415" confidence="0.9">Computer Systems Design and Related</naics>
      <keywords>
      <keyword>javascript</keyword>
      <keyword>ui</keyword>
      <keyword>open source</keyword>
      <keyword>frontend</keyword>
      <keyword>library</keyword>
      </keywords>
      <crawled>2026-08-12T09:00:00Z</crawled>
    </establishment>
    <establishment id="arxiv-transformer">
      <name>Attention Is All You Need</name>
      <url>https://arxiv.org/abs/1706.03762</url>
      <catalog>/page/arxiv-transformer</catalog>
      <domain>arxiv.org</domain>
      <snippet>We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.</snippet>
      <dewey code="006" confidence="0.94">Special computer methods</dewey>
      <naics code="5417" confidence="0.88">Scientific Research and Development</naics>
      <keywords>
      <keyword>transformer</keyword>
      <keyword>attention</keyword>
      <keyword>neural networks</keyword>
      <keyword>machine learning</keyword>
      <keyword>nlp</keyword>
      </keywords>
      <crawled>2026-08-12T09:00:00Z</crawled>
    </establishment>
    <establishment id="nist-cyber">
      <name>NIST Cybersecurity Framework</name>
      <url>https://www.nist.gov/cyberframework</url>
      <catalog>/page/nist-cyber</catalog>
      <domain>nist.gov</domain>
      <snippet>The NIST Cybersecurity Framework provides guidance for organizations to manage and reduce cybersecurity risk, organized around Identify, Protect, Detect, Respond, and Recover.</snippet>
      <dewey code="005" confidence="0.9">Computer programming, programs &amp; data</dewey>
      <naics code="5416" confidence="0.82">Management, Scientific, and Technical Consulting</naics>
      <keywords>
      <keyword>cybersecurity</keyword>
      <keyword>risk</keyword>
      <keyword>framework</keyword>
      <keyword>standards</keyword>
      <keyword>information security</keyword>
      </keywords>
      <crawled>2026-08-12T09:00:00Z</crawled>
    </establishment>
    <establishment id="w3-html">
      <name>HTML 5.2 — W3C Recommendation</name>
      <url>https://www.w3.org/TR/html52/</url>
      <catalog>/page/w3-html</catalog>
      <domain>w3.org</domain>
      <snippet>This specification defines the 5th major version, second minor revision of the core language of the World Wide Web: the Hypertext Markup Language (HTML).</snippet>
      <dewey code="006" confidence="0.86">Special computer methods</dewey>
      <naics code="5415" confidence="0.78">Computer Systems Design and Related</naics>
      <keywords>
      <keyword>html</keyword>
      <keyword>web standards</keyword>
      <keyword>markup</keyword>
      <keyword>browser</keyword>
      </keywords>
      <crawled>2026-08-12T09:00:00Z</crawled>
    </establishment>
  </sector>
</directory>

Companies by NAICS sector

Each sector has its own XML document. Other indexes can subscribe per industry.

Dewey class feeds

Robots.txt points crawlers at the sitemap. Alternate feed links are in the document head.