In: computer, technology.
Data processing
Data processing is the collecting and manipulation of data into the usable and desired form.
A list of command line tools for manipulating structured text data:
https://github.com/dbohdan/structured-text-tools
Tools
dasel: short for data-selector
Dasel allows you to query and modify data structures using selector strings
https://daseldocs.tomwright.me
https://github.com/TomWright/dasel
DuckDB
DuckDB is an in-process SQL OLAP database management system
https://duckdb.org
https://github.com/duckdb/duckdb
fx: function eXecution
Terminal JSON viewer
https://github.com/antonmedv/fx
grom
Make JSON greppable
Gron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute ‘path’ to it
https://github.com/tomnomnom/gron
jp: dead simple terminal plots
Dead simple terminal plots from JSON (or CSV) data
Bar charts, line charts, scatter plots, histograms and heatmaps
https://github.com/sgreben/jp
jql: a JSON query language
https://github.com/yamafaktory/jql
jql: JSON query processor with a Lispy syntax
https://github.com/cube2222/jql
miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
https://github.com/johnkerl/miller
https://miller.readthedocs.io
sq: swiss-army knife for data
sq is a free/libre open-source swiss-army knife to inspect, query, join, import, and export data
You can think of it as jq for relational data, whether that data is in a document or database
Supports SQL, JSON, CSV, XML
https://sq.io
https://github.com/neilotoole/sq
YouPlot
A command line tool that draw plots on the terminal
https://github.com/red-data-tools/YouPlot
Exploration
JSON visio
Seamlessly visualize your JSON data instantly into graphs
Paste - Import - Fetch!
https://jsonvisio.com
Jupyter Lab/ Notebook
https://github.com/jupyter
https://jupyter.org
Polynote
The polyglot notebook with first-class Scala support
Multi-language support - Polynote allows you to mix multiple languages in one notebook, while sharing definitions seamlessly between them
https://github.com/polynote/polynote
https://polynote.org
Libraries
Expr
Expression language for Go
Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not limited to, booleans). It is designed for simplicity, speed and safety.
https://github.com/antonmedv/expr
GJSON
https://github.com/tidwall/gjson
Get JSON values quickly - JSON parser for Go. JSON parse + JSON-ql language.
-
“children.1”
-
“friends.1.last”
-
“friends.#(age>45)#.last”
-
“children|@reverse”
JMESPath
JMESPath is a query language for JSON
https://github.com/jmespath
https://jmespath.org
JSON mask
Tiny language and engine for selecting specific parts of a JS object, hiding the rest
The main difference between JSONPath / JSONSelect and this engine is that JSON Mask preserves the structure of the original input object
https://github.com/nemtsov/json-mask
ObjectPath
The agile query language for semi-structured data. Written in Python.
https://github.com/adriank/ObjectPath
http://objectpath.org
Python JSONPath RW
A robust and significantly extended implementation of JSONPath for Python, with a clear AST for metaprogramming
https://github.com/kennknowles/python-jsonpath-rw