[Feature Request] Existing datagen pipleline performance enhancement (original) (raw)

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

Our current data generation pipeline can be more high-performance production readiness. We need to further enhance the existing pipelines, including: camel/datagen/source2synth, camel/datagen/self_instruct , camel/datagen/cotdatagen.py

1. production-ready performance and error handling

Solution

No response

Alternatives

No response

Additional context

No response