NIST Big Data Program

 

Welcome to NIST Big Data Public Working Group (NBD-PWG)!
Search  
 Home
 NBD-WG/Subgroups
   Charter
   Co-Chairs
   Guidelines
   All WG Meeting

 Documents
   Version 2 Final Docs
   Version 1 Final Docs
   Docs Repository
   Use Cases Listing
   Upload Document

 Registration
   New User
   Update Profile

 Points of Contact
   Wo Chang
     NIST / ITL
     Digital Data Advisor
     
   James St Pierre
     NIST / ITL
     Deputy Director
     
 

Use Cases and Requirements -- General
Use Cases V1.0 Submission [Requirements: UseCase | Summary | General | Gen+Ref | Gen+Ref+Gaps | Gen+Detail ]
(click M0180 to download full package and M0203 for full high-level use case descriptions)


Data Sources
General Requirement
1. needs to support reliable real time, asynchronize, streaming, and batch processing to collect data from centralized, distributed, and cloud data sources, sensors, or instruments
2. needs to support slow, bursty, and high throughput data transmission between data sources and computing clusters.
3. needs to support diversified data content ranging from structured and unstructured text, document, graph, web, geospatial, compressed, timed, spatial, multimedia, simulation, instrumental data.
Transformation
General Requirement
1. needs to support diversified compute intensive, analytic processing and machines learning techniques
2. needs to support batch and real time analytic processing
3. needs to support processing large diversified data content and modeling
4. needs to support processing data in motion (streaming, fetching new content, tracking, etc.)
Capability
General Requirement
1. needs to support legacy and advance software packages (subcomponent: SaaS)
2. needs to support legacy and advance computing platforms (subcomponent: PaaS)
3. needs to support legacy and advance distributed computing cluster, co-processors, I/O processing (subcomponent: IaaS)
4. needs to support elastic data transmission (subcomponent: networking)
5. needs to support legacy, large, and advance distributed data storage (subcomponent: storage)
6. needs to support legacy and advance programming executable, applications, tools, utilities, and libraries
Data Consumer
General Requirement
1. needs to support fast search (~0.1 seconds) from processed data with high relevancy, accuracy, and high recall
2. needs to support diversified output file formats for visualization, rendering, and reporting
3. needs to support visual layout for results presentation
4. needs to support rich user interface for access using browser, visualization tools
5. needs to support high resolution multi-dimension layer of data visualization
6. needs to support streaming results to clients
Security & Privacy
General Requirement
1. needs to support protect and preserve security and privacy on sensitive data
2. needs to support multi-level policy-driven, sandbox, access control, authentication on protected data
Lifecycle
General Requirement
1. needs to support data quality curation including pre-processing, data clustering, classification, reduction, format transformation
2. needs to support dynamic updates on data, user profiles, and links
3. needs to support data lifecycle and long-term preservation policy including data provenance
4. needs to support data validation
5. needs to support human annotation for data validation
6. needs to support prevention of data loss or corruption
7. needs to support multi-sites archival
8. needs to support persistent identifier and data traceability
9. needs to support standardize, aggregate, and normalize data from disparate sources
Others
General Requirement
1. needs to support rich user interface from mobile platforms to access processed results
2. needs to support performance monitoring on analytic processing from mobile platforms
3. needs to support rich visual content search and rendering from mobile platforms
4. needs to support mobile device data acquisition
5. needs to support security across mobile devices

 

Upcoming/Past Events

2nd NIST Big Data Workshop, NIST,
June 1 & 2, 2017
IEEE NBD-PWG Workshop, October 27, 2014
1st NIST Big Data Workshop, NIST, September 30, 2013

NIST is an agency of the
U.S. Commerce Department

Please send feedback to: bigdatainfo@nist.gov
Last updated: January 11, 2017

Privacy Police | Security Notices FOIA
Accessibility Statement | Disclaimer