GitHub - AsaiYusuke/jsonpath: A query library for retrieving part of JSON based on JSONPath syntax. (original) (raw)

AsaiYusuke/JSONPath

This Go library allows you to extract parts of a JSON object using the JSONPath query syntax.

The core JSONPath syntax supported by this library is based on:

• Stefan Gössner's JSONPath - XPath for JSON
• Christoph Burgmer's json-path-comparison
• JSONPath Internet Draft Development

Note

For details on syntax compatibility with other libraries, see 📝 my comparison results.

Table of Contents

• Getting started
• Basic design
• How to use
  • Retrieve one-time or repeated
  • Error handling
  • Function syntax
  • Accessing JSON
• Differences
• Benchmarks
• Project progress

Getting started

Install

go get github.com/AsaiYusuke/jsonpath/v2

Simple example

package main

import ( "encoding/json" "fmt"

"github.com/AsaiYusuke/jsonpath/v2" )

func main() { jsonPath, srcJSON := $.key, {"key":"value"} var src any json.Unmarshal([]byte(srcJSON), &src) output, _ := jsonpath.Retrieve(jsonPath, src) outputJSON, _ := json.Marshal(output) fmt.Println(string(outputJSON)) // Output: // ["value"] }

Basic design

Streamlined Development

• The JSONPath syntax parser is implemented using PEG, which helps keep the source code simple and maintainable.
• Robust unit tests are provided to prevent bugs and ensure consistent results.

User-Friendly Interface

• The library provides comprehensive error types, making it easy for users to handle errors appropriately.

High Compatibility

• The library incorporates consensus behaviors from Christoph Burgmer's json-path-comparison, ensuring high compatibility with other implementations.

How to use

* Retrieve one-time or repeatedly

The Retrieve function extracts values from a JSON object using a JSONPath expression:

output, err := jsonpath.Retrieve(jsonPath, src)

📝 Example

The Parse function returns a parser function that checks the JSONPath syntax in advance. You can use this parser function to repeatedly extract values with the same JSONPath:

parsed, err := jsonpath.Parse(jsonPath) output1, err1 := parsed(src1) output2, err2 := parsed(src2)

📝 Example

parser function

• The parser function accepts an optional second argument: a pointer to a []any buffer.
• When omitted or nil, a new slice is allocated on each call and returned.
• When provided (non-nil), the buffer will be reset to length 0 and filled directly, enabling reuse without extra allocations.

Example of buffer reuse:

parsed, _ := jsonpath.Parse("$[]") buf := make([]any, 0, 4) args := [][]any{&buf} out1, _ := parsed([]any{1}, args...) // writes into buf -> [1] out2, _ := parsed([]any{2}, args...) // reuses the same buf -> now [2] fmt.Println(out1) fmt.Println(out2) fmt.Println(buf) // Output: // [2] // [2] // [2]

📝 Example

Note: Do not share the same buffer across goroutines concurrently.

* Error handling

If an error occurs during API execution, a specific error type is returned. The following error types help you identify the cause:

Syntax errors from Retrieve or Parse

Runtime errors from Retrieve or parser functions

Type checking makes it easy to determine which error occurred.

import jsonpath "github.com/AsaiYusuke/jsonpath/v2" import errors "github.com/AsaiYusuke/jsonpath/v2/errors"

_, err := jsonpath.Retrieve(jsonPath, srcJSON) switch err.(type) { case errors.ErrorMemberNotExist: fmt.Printf("retry with other srcJSON: %v", err) // handle or continue case errors.ErrorInvalidArgument: return nil, fmt.Errorf("specified invalid argument: %v", err) }

* Function syntax

You can use user-defined functions to format results. The function syntax is appended after the JSONPath expression.

There are two types of functions:

Filter function

A filter function applies a user-defined function to each value in the result, transforming them individually.

📝 Example

Aggregate function

An aggregate function combines all values in the result into a single value.

📝 Example

* Accessing JSON

Instead of retrieving values directly, you can obtain accessors (Getters / Setters) for the input JSON. These accessors allow you to update the original JSON object.

Enable this feature by calling Config.SetAccessorMode().

📝 Example

Accessor limitations

Setters are not available for some results, such as when using function syntax in the JSONPath.

Accessor operations follow Go's map/slice semantics. If you modify the structure of the JSON, be aware that accessors may not behave as expected. To avoid issues, obtain a new accessor each time you change the structure.

Differences

Some behaviors in this library differ from the consensus of other implementations. For a full comparison, see 📝 this result.

These behaviors may change in the future if more appropriate approaches are found.

Character types

The following character types are allowed for identifiers in dot-child notation:

Printable symbols (except hyphen and underscore) can be used by escaping them.

JSONPath : $.abc\.def
srcJSON  : {"abc.def":1}
Output   : [1]

Wildcard in qualifier

Wildcards in qualifiers can be specified as a union with subscripts.

JSONPath : $[0,1:3,*]
srcJSON  : [0,1,2,3,4,5]
Output   : [0,1,2,0,1,2,3,4,5]

Regular expression

Regular expression syntax follows Go's regular expression rules. In particular, you can use "(?i)" to make the regular expression case-insensitive.

JSONPath : $[?(@=~/(?i)CASE/)]
srcJSON  : ["Case","Hello"]
Output   : ["Case"]

JSONPaths in the filter-qualifier

JSONPaths that return a value group cannot be used with a comparator or regular expression. However, you can use these syntaxes for existence checks.

• comparator example (error)

JSONPath : $[?(@..x == "hello world")]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax

• regular expression example (error)

JSONPath : $[?(@..x=~/hello/)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax

• existence check example

JSONPath : $[?(@..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"b":{"x":"hello world"}}]

If a JSONPath filter begins with the Root, it performs a whole-match operation if any match is found.

JSONPath : <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">[</mo><mo stretchy="false">?</mo><mo stretchy="false">(</mo></mrow><annotation encoding="application/x-tex">[?(</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">[</span><span class="mclose">?</span><span class="mopen">(</span></span></span></span>..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"a":1},{"b":{"x":"hello world"}}]

Benchmarks

Benchmark results for various Go JSONPath libraries (measured by myself) are available in the following repository:

• JSONPath Benchmark

Project progress

• Syntax
  • Identifier
    * identifier in dot notation
    * identifier in bracket notation
    * wildcard
    * multiple identifiers in brackets
    * recursive retrieval
  • Qualifier
    * index
    * slice
    * wildcard
    * Filter
    * logical operations
    * comparators
    * JSONPath retrieval in filter
    * script
  • Function
    * filter
    * aggregate
  • Refer to the consensus behaviors
• Architecture
  • PEG syntax analysis
  • Error handling
  • Function support
  • JSON accessors
• Go language manner
  • retrieve with an object unmarshaled to interface
  • retrieve with the json.Number type
• Source code
  • Release version
  • Unit tests
    * syntax tests
    * benchmarks
    * coverage >80%
  • Examples
  • CI automation
  • Documentation
    * README
    * API documentation
  • comparison results (local)
• Development status
  • requirements and functional design
    * Decided to follow a standard or reference implementation for JSONPath syntax
  • design-based coding
  • testing
  • documentation
• Future ToDo
  • Go language affinity
    * retrieve with an object unmarshaled to struct
    * retrieve with struct tags
    * retrieve with user-defined objects