Artificial Intelligence Standards

AGENCY:

National Institute of Standards and Technology, U.S. Department of Commerce.

ACTION:

Notice; Request for Information (RFI).

SUMMARY:

The February 11, 2019, Executive Order on Maintaining American Leadership in Artificial Intelligence (AI) directs the National Institute of Standards and Technology (NIST) to create a plan for Federal engagement in the development of technical standards and related tools in support of reliable, robust, and trustworthy systems that use AI technologies (Plan). This notice requests information to help NIST understand the current state, plans, challenges, and opportunities regarding the development and availability of AI technical standards and related tools, as well as priority areas for federal involvement in AI standards-related activities. To assist in developing the Plan, NIST will consult with Federal agencies, the private sector, academia, non-governmental entities, and other stakeholders with interest in and expertise relating to AI.

DATES:

Comments in response to this notice must be received May 31, 2019.

ADDRESSES:

Written comments in response to this RFI may be submitted by mail to AI-Standards, National Institute of Standards and Technology, 100 Bureau Drive, Stop 2000, Gaithersburg, MD 20899. Online submissions in electronic form may be sent to ai_standards@nist.gov. Submissions may be in any of the following formats: HTML, ASCII, Word, RTF, or PDF. Please cite “RFI: Developing a Federal AI Standards Engagement Plan” in all correspondence. All relevant comments received by the deadline will be posted at https://www.nist.gov/topics/artificial-intelligence/ai-standards and regulations.gov without change or redaction, so commenters should not include information they do not wish to be posted (e.g., personal or confidential business information). Comments that contain profanity, vulgarity, threats, or other inappropriate language or content will not be posted or considered.

FOR FURTHER INFORMATION CONTACT:

For questions about this RFI contact: Elham Tabassi, NIST, MS 8900, 100 Bureau Drive, Gaithersburg, MD 20899, telephone (301) 975-5292, email elham.tabassi@nist.gov. Please direct media inquiries to NIST's Public Affairs Office at (301) 975-NIST.

SUPPLEMENTARY INFORMATION:

Genesis of the Plan for Federal Engagement in Artificial Intelligence Standards

The Executive Order (E.O.) on AI states that “[c]ontinued American leadership in AI is of paramount importance to maintaining the economic and national security of the United States and to shaping the global evolution of AI in a manner consistent with our Nation's values, policies, and priorities.” Accordingly, Section 1 of the E.O. calls for a coordinated Federal Government strategy, the American AI Initiative, and notes that the U.S. must drive development of appropriate AI technical standards in order to enable the creation of new AI-related industries and the adoption of AI by today's industries. This can be achieved through the work and partnership of industry, academia, and government.

Section 1(d) of the E.O. states that the U.S. must foster public trust and confidence in AI technologies and protect civil liberties, privacy, and American values in their application in order to fully realize the potential of AI technologies for the American people.

Section 2(d) of the E.O. directs Federal agencies to ensure that technical standards minimize vulnerability to attacks from malicious actors and reflect Federal priorities for innovation, public trust, and public confidence, and to develop international standards to promote and protect those priorities.

Section 6(d) of the E.O. directs the Secretary of Commerce, acting through the Director of NIST, to issue a Plan for Federal engagement in the development of technical standards and related tools in support of reliable, robust, and trustworthy systems that use AI technologies. It further directs NIST to lead the development of the Plan with participation from relevant agencies, as determined by the Secretary of Commerce.

Approach for Developing This Plan

NIST will develop the Plan in a manner that fulfills the objectives of the E.O. and is consistent with relevant provisions of the Office of Management and Budget (OMB) Circular A-119, “Federal Participation in the Development and Use of Voluntary Consensus Standards and in Conformity Assessment Activities,” and NIST's mission to promote U.S. innovation and industrial competitiveness. NIST has a special interest in advancing the development and use of standards relied upon by all sectors of the economy and society, recognizing that the vast majority of standards are developed through a voluntary process led by the private sector.

NIST will be informed through an open process that will include this RFI and other opportunities, such as a public workshop, to provide input. NIST expects to develop a draft Plan on which it will seek comment from the public and Federal agencies. Information about this effort, including ways to provide input, and future steps, will be available at https://www.nist.gov/topics/artificial-intelligence/ai-standards.

Goals of This Request for Information

Timely and fit-for-purpose AI technical standards—whether developed by national or international organizations—will play a crucial role in the development and deployment of AI technologies, and will be essential in building trust and confidence about AI technologies and for achieving economies of scale.

NIST seeks to understand the:

• Current status and plans regarding the availability, use, and development of AI technical standards and tools in support of reliable, robust, and trustworthy systems that use AI technologies;
• Needs and challenges regarding the existence, availability, use, and development of AI standards and tools; and
• The current and potential future role of Federal agencies regarding the existence, availability, use, and development of AI technical standards and tools in order to meet the nation's needs.

For purposes of this Plan, AI technologies and systems are considered to be comprised of software and/or hardware that can learn to solve complex problems, make predictions or solve tasks that require human-like sensing (such as vision, speech, and touch), perception, cognition, planning, learning, communication, or physical action. Examples are wide-ranging and expanding rapidly. They include, but are not limited to, AI assistants, computer vision systems, automated vehicles, unmanned aerial systems, voicemail transcriptions, advanced game-playing software, facial recognition systems as well as application of AI in both Information Technology (IT) and Operational Technology (OT).

Responding to This Request for Information

The scope of this RFI includes AI technical standards and related tools regardless of origin or use. Respondents may define “standards” as they desire, indicating clearly what they mean when using the term. AI technical standards and related tools should include those necessary or helpful to reduce barriers to the safe testing and deployment of AI and to support reliable, robust, and trustworthy systems that use AI technologies.

Respondents may define tools as broadly or as narrowly as they wish. They should indicate clearly what they mean when using specific terms (e.g., practices, datasets, guidelines). An illustrative, non-exclusive list of standards-related tools includes:

• Test tools (e.g., executable test code) for conformance testing, performance testing, stress testing, interoperability testing, and other purposes;
• Use cases;
• Reference data and datasets;
• Reference implementations; and
• Training programs.

Where this RFI uses the term “organizations,” it refers to private, public, and non-profit bodies, and includes both national and international organizations. If desired, commenters may provide information about: The type, size, and location of their organization(s); and whether their organization develops AI technology and related tools; uses or potentially uses AI technology and related tools; and/or participates in the development of AI standards or related tools. Provision of such information is optional and will not affect NIST's full consideration of the comment.

Comments containing references—including specific standards and related tools—studies, research, and other empirical data that are not widely published (e.g., available on the internet) should include paper or electronic copies of those materials, unless they are restricted due to copyright or are otherwise proprietary. In those cases, NIST encourages respondents to provide clear descriptions and designations of those references. Do not include in comments or otherwise submit any information deemed to be proprietary, private, or in any way confidential, as all comments relevant to this RFI topic area that are received by the deadline will be made available publicly at https://www.nist.gov/topics/artificial-intelligence/ai-standards and regulations.gov.

The following list of topics covers the major areas about which NIST seeks information. This list is not intended to limit the topics that may be addressed by respondents, who may provide information about any topic which would inform the development of the Plan. Possible topics, subdivided by area, are:

AI Technical Standards and Related Tools Development: Status and Plans

1. AI technical standards and tools that have been developed, and the developing organization, including the aspects of AI these standards and tools address, and whether they address sector-specific needs or are cross-sector in nature;

2. Reliable sources of information about the availability and use of AI technical standards and tools;

3. The needs for AI technical standards and related tools. How those needs should be determined, and challenges in identifying and developing those standards and tools;

4. AI technical standards and related tools that are being developed, and the developing organization, including the aspects of AI these standards and tools address, and whether they address sector-specific needs or are cross sector in nature;

5. Any supporting roadmaps or similar documents about plans for developing AI technical standards and tools;

6. Whether the need for AI technical standards and related tools is being met in a timely way by organizations; and

7. Whether sector-specific AI technical standards needs are being addressed by sector-specific organizations, or whether those who need AI standards will rely on cross-sector standards which are intended to be useful across multiple sectors.

8. Technical standards and guidance that are needed to establish and advance trustworthy aspects (e.g., accuracy, transparency, security, privacy, and robustness) of AI technologies.

Defining and Achieving U.S. AI Technical Standards Leadership

9. The urgency of the U.S. need for AI technical standards and related tools, and what U.S. effectiveness and leadership in AI technical standards development should look like;

10. Where the U.S. currently is effective and/or leads in AI technical standards development, and where it is lagging;

11. Specific opportunities for, and challenges to, U.S. effectiveness and leadership in standardization related to AI technologies; and

12. How the U.S. can achieve and maintain effectiveness and leadership in AI technical standards development.

Prioritizing Federal Government Engagement in AI Standardization

13. The unique needs of the Federal government and individual agencies for AI technical standards and related tools, and whether they are important for broader portions of the U.S. economy and society, or strictly for Federal applications;

14. The type and degree of Federal agencies' current and needed involvement in AI technical standards to address the needs of the Federal government;

15. How the Federal government should prioritize its engagement in the development of AI technical standards and tools that have broad, cross-sectoral application versus sector- or application-specific standards and tools;

16. The adequacy of the Federal government's current approach for government engagement in standards development, which emphasizes private sector leadership, and, more specifically, the appropriate role and activities for the Federal government to ensure the desired and timely development of AI standards for Federal and non-governmental uses;

17. Examples of Federal involvement in the standards arena (e.g., via its role in communications, participation, and use) that could serve as models for the Plan, and why they are appropriate approaches; and

18. What actions, if any, the Federal government should take to help ensure that desired AI technical standards are useful and incorporated into practice.