[Feature Request] add load/save function in StaticDataset (original) (raw)

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

Requested for the Loong project: please add load and save functions for StaticDataset. I can see that currently the load function is enabled, please add a save function, either to local or huggingface hub. This helps Loong teams align with the dataset format. I am wondering if having something like load_dataset or load_loong_dataset as a standalone function to mimic the convention of huggingface is worth considering.

Please also take care of the JSON serialization problem.

Solution

Possible reference code: https://github.com/camel-ai/loong_private/blob/master/domain/logic/loong_logic/data.py

Alternatives

No response

Additional context

No response