Skip to content

siara-cc/csv_parser_npmjs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Multi-level CSV (CSV_ML)

This is an Advanced Parser for CSV (Comma-separated-value), TSV (Tab-separated-value), TDV (Tab-delimited-value) or even files with custom delimiters such as the Pipe symbol (|). It supports streams and pull parsing for handling huge data files. It also supports comments and empty lines within the delimited files for annotation.

In addition to parsing regular tabular formats with or without header, it can also parse structured delimited files and convert them to XML DOM or JSON objects.

Promo picture

This project proposes the idea of using CSV format for defining structured relational data in additional to tabular data. The idea is nick named CSV_ML (Multi-level CSV). CSV_ML attempts to provide a simple unambiguous format for representing structured data that includes schema definition.

Installation

For using with Node.js, install using npm install csv_ml_parser.siara.cc. For using it on a browser, include csv_parser.js provided in this repository.

Usage

Basic usage

Instantiate CSV_ML_Parser class, and parse any CSV or Multi-Level CSV into Javascript object:

var csv_ml = require('csv_ml_parser.siara.cc');
var csv_ml_parser = new csv_ml.CSV_ML_Parser("csv_ml,1.0\n" +
                        "Contact,name,occupation,company\n" +
                        " Telephone,type,number\n" +
                        "end_schema\n" +
                        "Contact,Jason Blum,Founder/CEO,Blumhouse Productions\n" +
                        " Telephone,mobile,202-555-0168\n" +
                        " Telephone,office,202-555-0185\n" +
                        "Contact,Kevin Beggs,Chairman,Lionsgate TV Group\n" +
                        " Telephone,mobile,617-555-0148\n" +
                        " Telephone,office,617-555-0162");
var ret = csv_ml_parser.parse("jso", false);
if (csv_ml_parser.ex.display_exceptions()) { /* Handle error */ return };
console.log(JSON.stringify(ret, null, ' '));

Try out different pre-defined examples online at http://siara.cc/csv_ml/csv_ml_js.html

Output

{
 "Contact": [
  {
   "name": "Jason Blum",
   "occupation": "Founder/CEO",
   "company": "Blumhouse Productions",
   "Telephone": [
    {
     "type": "mobile",
     "number": "202-555-0168"
    },
    {
     "type": "office",
     "number": "202-555-0185"
    }
   ]
  },
  {
   "name": "Kevin Beggs",
   "occupation": "Chairman",
   "company": "Lionsgate TV Group",
   "Telephone": [
    {
     "type": "mobile",
     "number": "617-555-0148"
    },
    {
     "type": "office",
     "number": "617-555-0162"
    }
   ]
  }
 ]
}

Advantages over XML and JSON

CSV_ML

  • saves storage space (about 50% compared to JSON and 60-70% compared to XML)
  • increases data transfer speeds
  • is faster to parse compared to XML and JSON
  • allows full schema definition and validation
  • makes schema definition simple, lightweight and in-line compared to DTD or XML Schema
  • recognizes standard data types including text (varchar), integer, real, date, datetime
  • allows database binding
  • can be used in EAI (Application Integration) for import and export of data
  • is simpler to parse, allowing data to be available even in low memory devices

Applications

  • Enterprise Application Integration (EAI)
  • Lightweight alternative to JSON or XML in Three-tier architecture
  • Alternative to XML in transfer of data using AJAX
  • Data storage and transfer format for embedded platforms such as Arduino and Raspberry PI.
  • Data storage and transfer format for mobile/tablet devices based on Android, Windows or iOS.
  • Data transfer format for spreadsheets as Tab delimited values through clipboard or otherwise.

For complete documentation and examples, download Multi-level nested CSV.pdf

Examples

The examples given in the documentation are available as demo applications:

For running Javascript and Java Applet demos, you may have to change security settings in your browser.

The given demos convert between CSV, TDV. XML and JSON (CSV to XML DOM, CSV to JSON, TDV to XML DOM, TDV to JSON, XML to CSV). It is basically a CSV TDV TSV to JSON XML Convertor. It also demonstrates how database binding can be achieved using SQLite db.

Screenshots

Example 1.1: Conventional CSV













Example 1.2: Conventional CSV without Header













Example 1.3 and 1.4: Conventional CSV with Header and Node name

























Example 1.5: Multiple nodes under root













Example 2.1: Multiple level CSV data













Example 2.2: Multiple level CSV data with siblings













Example 3.1: Node attributes and content













Example 3.2: Node content













Example 3.3: Quote handling













Example 3.4: Inline comments and empty lines













Example 3.5: Changing root node













































Example 3.6: Namespaces































Example 3.7: Re-using node definitions











Example 4.1: Specifying Type and Length



Example 4.2: Default value



Example 4.3: Null values





Example 4.4: Precision and Scale



Example 4.5: Date and Time



Example 4.6: Special column id



Example 4.7: Special column parent_id



Retrieving data for Example 2.1



Retrieving data for Example 1.5



Contact

Create issue here or contact [email protected] for any queries or feedback.