24153 Destination Databricks: Fix support for external tables on S3 by suhomud · Pull Request #24657 · airbytehq/airbyte (original) (raw)
that may beclosed by this pull request
24153 Destination Databricks: Update s3 source external 24153 Destination Databricks: Fix support for external tables on S3
btkcodedev pushed a commit to btkcodedev/airbyte that referenced this pull request
24153 Destination Databricks: Update s3 source external
24153 Destination Databricks: Fixed databricks.md formatting
24153 Destination Databricks: Update the version
auto-bump connector version
Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
marcosmarxm added a commit that referenced this pull request
Initial commit
Add stream top_reviews
Add new stream: privacy_data
add new stream: privacy user, refactor
Add new streams, update docs, update definitions
EOF
Update spec.yaml
fix date_from, optimize manifest, update docs
delete privacy streams, refactor webhooks
add pagination
add streams raw_data, email_analytics
add streams unsubscribers, webhook_events
Fix pagination
revert doc
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
Revert "remove top_reviews and reviews_bottomline, Make common paginator"
This reverts commit 6615288b6d2441d9fdde2d927a5eb77b5d788442.
remove streams top_reviews, reviews_bottomline
make common base and extractor
refactor, Checkpoint for full test passed
refactor base, requester, paginator
remove extra paginator property
Update oss_catalog.json
fix conflicts
fix conflict oss
fix conflict oss
Add stream top_reviews
Add new stream: privacy_data
add new stream: privacy user, refactor
Add new streams, update docs, update definitions
EOF
Source Stripe: expand tiers when syncing plans (#19906)
Expand tiers when syncing plans
Update Plans stream schema
Re-bump
Bump
Remove string as a plan schema
bump
Update changelog
Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
airbyte-ci: improve metadata validation perfs (#25107)
metadata service: upload metadata files to gcs (#25115)
metadata: cleanup workflows (#25105)
Source Stripe: generate seed source file (#25113)
Expand tiers when syncing plans
Update Plans stream schema
Re-bump
Bump
Remove string as a plan schema
bump
Update changelog
update source spec
Automated Change
Co-authored-by: Visnu Pitiyanuvath visnup@gmail.com Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com
Connector builder: set pages and slices limits (#25121)
Set limits
refactor and add unit tests
Update as per comments
🤖 Bump patch version of Airbyte CDK
Bump Airbyte version from 0.43.1 to 0.43.2
source-postgres works with bigger datasets (#25138)
create metadata files for all connectors (#24964)
fix date_from, optimize manifest, update docs
Update spec.yaml
🚨🚨 Source Amplitude: migrate to YAML (#22362)
Source Amplitude: migrate to YAML
Source Amplitude: enable high strictness level
Source Amplitude: update docs
Source Amplitude: update cursor field + expected records
Source Amplitude: logging refactor
Source Amplitude: refactor extractors
Source Amplitude: refactor extractors
Source Amplitude: migrate to beta low-code
Source Amplitude: add docstring
Source Amplitude: fix unit tests
Source Amplitude: update expected records
Source Amplitude: fix response filter action
Source Amplitude: rename typo
Source Amplitude: fix migration
Source Amplitude: update docs
Source Amplitude: fix migration
Source Amplitude: update expected recrods
Source Amplitude: bump manifest version
Automated Change
Source Amplitude: Update http -> https
Source Amplitude: tests fix + pin cdk version
auto-bump connector version
Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Support: update expected records (#25058)
Source Zendesk Support: skip stream
Add expected records for tags stream
Automated Change
Co-authored-by: arsenlosenko arsenlosenko@users.noreply.github.com
Source Surveymonkey: Fix spec.json required fields and update schema for surveys and survey_responses (#25080)
Fixes required fields on SurveyMonkey UI
fix integration test schema update for surveys and survey response
fix expected records
update dockerfile and docs
auto-bump connector version
Co-authored-by: Ali Hassan alihassan01264@gmail.com Co-authored-by: Ali Hassan 62539376+Alihassanc5@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add additional destination-bigquery datacenters (#25122)
Add additional bigquery datacenters
bump dockerfile
revert source BQ bump
docs
builds depend on normalization now...
no -base
auto-bump connector version
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Bump versions in atelier dockerfile (#25125)
Bump versions in atelier dockerfile
fix new version
temporarily skip build and publish
reset
skip some steps
Revert "skip some steps"
This reverts commit e034d10.
Source: Amazon Ads - Add pk validator for reports when expected pk is not returned (#25146)
Add pk validator for reports when expected pk is not returned
Update PR number
Fix formating
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉Source Twilio: New Stream Conversation Participants (#23995)
Source Twilio: New Stream Conversation Participants
fix: format schema and fix to incremental stream
refactor: Add newline
fix: remove unwanted file
fix: schema fixes
fix: Avoid acceptance test bypass for full testing
fix: Fixed incremental behaviour, Updated docs, Updated VersionTag
fix: Fixed incremental behaviour, Updated docs, Updated VersionTag
fix: Fixed syntax, Updated docs
fix: schema, update docs
fix: Changed parent stream, expected record fixes, connector version bump
chore: EOL fix
fix expected records for conversations participant
Update Twilio version to 0.5.0
update twilio version to 0.5.0
remove conversation_participants from state file
add lines and add eof
update twilio version to 0.5.0 in source spec
removed conversation_participants from abnormal state
remove extra lines
update doc connector version
update json schema draft version
fix doc conflict
add expected records
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Iterable: update expected records (#25144)
Source Facebook Pages: update insight metrics request params (#25143)
Source Facebook Pages: update insight metrics request params
Update Dockerfile and changelog
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
fix strict-encrypt metadata dockerRepository (#25159)
Source Slack: update expected records (#25150)
dagger: upgrade to 0.5.0 (#25171)
dagger: upgrade to 0.5.0
clean class ConnectorPackageInstall(Step):
Fixed missing punctuations (#25151)
Update README.md
I gave a better description to the Airbyte Cloud section, converted a few passive words to active, and added a missing comma.
- Update README.md
I noticed several punctuation marks were missing, so I fixed them. I also want to use this opportunity to draw get the attention of the docs team. I would love to join Airbyte as a technical writer and be a part of the team. I am more than willing to handle these tasks, even if the organization does not have the current budget to onboard a new technical writer.
Also, I filled the form to participate in the Airbyte Writer Program. I hope the team reviews my topic and allow me contribute to the data insight and tutorial articles. Cheers!
- Update README.md
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
refactor source harness and GitHub action to add source-mysql (#25036)
refactor to add source-mysql
refactor to add source-mysql
refactor to add source-mysql
test
Automated Change
fix if condition
try multiline
new db
add mysql catalogs
Co-authored-by: rodireich rodireich@users.noreply.github.com
- change google analytics dockerfile order to pip install source (#23082)
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
Always show client cert and key (#25131)
make CA cert and key always show
make same change for mysql
update readmes
update strict-encrypt version and expected spec
fix expected spec for mysql strict encrypt
auto-bump connector version
manually bump seed for postgres and mysql
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🐛 Source File: fix column data types for numerical values (#23759)
Fix column data types for numerical values
Update unit tests
Update Dockerfile version
update dockerfile version for source-file-secure
Update source-file in seed file
Update doc
Update source_specs.yaml
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
connectors.mdgains Github labels for connector issues (#25182)connectors.mdgains github labelAutomated Commit - Formatting Changes
comment
Automated Commit - Formatting Changes
Automated Change
Co-authored-by: evantahler evantahler@users.noreply.github.com
- Housekeeping: Remove unused workflows. (#25191)
This is no longer used with Move's move to Zenhub.
increase sentry heartbeat - 43200->64800 (#25185)
increase sentry heartbeat - 43200->64800
Automated Change
Co-authored-by: colesnodgrass colesnodgrass@users.noreply.github.com
Bump Airbyte version from 0.43.2 to 0.44.0
dagger CI: auth to dockerhub (#25196)
auth to dockerhub sometime
map docker hub username and password to env var
Revert "auth to dockerhub sometime"
This reverts commit d82bf88.
- Revert "Revert "auth to dockerhub sometime""
This reverts commit bc9bc61.
login to dockerhub
login to dockerhub
revert changes
airbyte-ci: orchestrate the upload of modified metadata.yaml files to GCS (#25044)
Source Hubspot: update expected records (#25152)
Source Hubspot: update expected records
Update expected records
Source Github: update expected records (#25148)
Source Github: update expected records
Update expected records
Move teams expected records to previous place
Source Pipedrive: update expected records (#25199)
Source Pipedrive: update expected records
Update expected records
Removed expected record that is not needed
Source Zendesk Support: update expected records (#25206)
Source Twilio: update expected records (#25200)
connector builder: Emit message at start of slice (#25180)
Move condition for yielding the slice message to an overwritable method
Automated Commit - Formatting Changes
yield the slice log messages
same for incremental
refactor
Revert "refactor"
This reverts commit c594365.
move flag from factory to source
set the flag
remove debug print
halfmock
clean up
Add a test for a single page
Add another test
Pass the flag
rename
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump patch version of Airbyte CDK
feat(source-bing-ads): add dimensional columns for AccountPerformanceReport
feat(source-bing-ads): add dimensional columns for AdGroupPerformanceReport
feat(source-bing-ads): add dimensional columns for AdPerformanceReport
feat(source-bing-ads): add dimensional columns for KeywordPerformanceReport
feat(source-bing-ads): add dimensional columns for CampaignPerformanceReport
docs(source-bing-ads): update changelog
chore(source-bing-ads): bump version in Dockerfile
test(source-bing-ads): allow extra fields for basic read
Revert "test(source-bing-ads): allow extra fields for basic read"
This reverts commit 961f0d6.
test(source-bing-ads): ignore community contributed fields
fix: updated expected_records of report streams
auto-bump connector version
Automated Change
Co-authored-by: Rachel RIZK rachel.rizk@backmarket.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com Co-authored-by: sajarin sajarin@users.noreply.github.com
Source Facebook Marketing: fix data retention period validation (#25204)
#1678 source Facebook Marketing: fix data retention period validation
#1678 source fb marketing: add validation for the date range in the future
#1678 source fb marketing: another fix
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add OSS getting started redirect (#25213)
Add is_optional flag to connector tests (#25211)
Run doc deploys on docosaurus changes (#25215)
Co-authored-by: Topher Lubaway asimplechris@gmail.com
Fix manifest_declarative_source + add unit tests (#25217)
Fix + unit test
Add a test with pagination
Add a test with partition router
Make sure _fetch_next_page is called with the right arguments
Automated Commit - Formatting Changes
pagination with partitions
refactor
clean up
format
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump patch version of Airbyte CDK
Document the config for docker workspace cleanup (#25210)
delete privacy streams, refactor webhooks
add pagination
add streams raw_data, email_analytics
add streams unsubscribers, webhook_events
Fix pagination
revert doc
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
Revert "remove top_reviews and reviews_bottomline, Make common paginator"
This reverts commit 6615288b6d2441d9fdde2d927a5eb77b5d788442.
remove streams top_reviews, reviews_bottomline
make common base and extractor
refactor, Checkpoint for full test passed
refactor base, requester, paginator
remove extra paginator property
Cleanup some logging in connectors (#25227)
🎉 New Source: Teradata (#24221)
add teradata template
add teradata environment client
add teradata source connector
add docs
fix lowercase letter
fix compilation error & config test
fix timestamp
generate seed
Co-authored-by: Prateek Mukhedkar 123108018+prateekmukhedkar@users.noreply.github.com
Add metadata gcs sensor to Dagster (#25055)
Add solution skeleton
Add metadata blob resource
Wire up sensor
Fix report
Materialize metadata definitions from folder
Move metadata to own file
Run black
Add PR comments
Remove dockerhub from action
🎉Source Stripe: New Stream - Subscription Schedule (#23624)
bump connector version
Source stripe: New Stream Subscription Schedule
fix: schema fixes, updated docs, updated versionTag
fix: test
Update stripe.md
fix: Add empty stream
schema fix, version bump, source update
unwanted line fix
Resolve comments
fix schemas
add expected records and update schema
🎉Source Twilio: New Stream Conversation Messages (#22875)
bump connector version
Source Twilio: New Stream Conversation Messages
fix: Adding correction to syntax
fix: schema errors, incremental stream fix
fix: schema, updated docs, dockerVersion, removed unwanted files
Update twilio.md
fix schema, add expected records and update stream
remove extar empty line
format files
fix schema for conversation messages
fix conversation participants schema
remove conversation participants from incr ctg
remove unused logging
update schema
Update source_definitions.yaml
Update jira.md
Update expected_records.jsonl
Update twilio.md
Update twilio.md
Update twilio.md
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
remove json schema from integration test
add schema json
Update jira.md
Update stripe.md
Update source_definitions.yaml
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add compatibility guide to sidebar (#25224)
Shuffled
destination-jdbcintobasesfolder (#25120)Testing changes for reshuffle
Resolves dependency/race-condition with base classes and docker creation
Removes dockerfile
Updates java connectors gradle dependencies
Run ./gradlew :airbyte-config:init:processResources (#25250)
add note on security self-check to security.md (#25235)
Source Copper: Add opportunities to streams (#24824)
Add opportunities to copper streams
Fix version
update schema
update versions and fix schema formatting
auto-bump connector version
Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
low-code: Clean up SessionTokenAuthenticator interface (#25086)
Username and session token are optional fields
update
Add titles, descriptions, and examples
Automated Commit - Formatting Changes
fix a small typo
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump minor version of Airbyte CDK
Add logging functionality to understand Xmin wraparound (#25220)
Add logging functionality
- Add logging for DB version (all JDBC sources)
- Add logging for Xmin values
Automated Change
Renaming + comments
More comments
Bump docker version + docs
auto-bump connector version
Update failed source_specs.yaml
Co-authored-by: akashkulk akashkulk@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
metadata: create specs secret mask (#25201)
Connector builder: Authentication documentation (#25112)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
review comments
review comments
Connector builder: Partitioning documentation (#25209)
partitioning documentation
review comments
Source S3: Add start date filter for files (#25010)
Source S3: Add start date filter for files
Source S3: add docs
Source S3: add unittest
Source S3: add unittest
Source S3: add unittest
Source S3: Fix spec test
Source S3: bump version
Source S3: fix tests
Source S3: fix description
auto-bump connector version
Source S3: refactor start_date filtering
Source S3: update setup
Source S3: serialize state for cache
Source S3: refactor skip file filter
Source S3: bump version + update docs
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Chat: update expected records (#25271)
Source Sentry: update expected records (#25270)
Source S3: handle block size related errors (#25067)
Source S3: handle pyarrow block size errors
Source S3: bump version
Automated Change
Source S3: fix null field check
Revert "Automated Change"
This reverts commit dc707f7.
Automated Change
Source S3: bump version + update docs
auto-bump connector version
Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Support: update expected records (#25272)
Add Dagster Deployment Action skeleton (#24929)
Add code skeleton
Remove pr deployments
remove connectors change
Add action git ignore
Update YAML to deploy properly
Add newline at eof
Setup python-executable deploy
Add temporary script
Update readme
Remove github workflow
Add dagger pipeline for deploy
Add github action
Update readme
Remove comment
Move script into dagger pipeline
Add run test helper and skip test logic
Have the dagster orchestrator properly generate the setup.py file
Remove deploy script
Address PR comments
Update lock
Automated Commit - Formatting Changes
Co-authored-by: bnchrch bnchrch@users.noreply.github.com
🎉 Source Facebook Marketing: Increase pause interval time after reach MIN_RATE or MAX_RATE. (#22990)
Change min rate and increase time in pauses
Update Dockerfile
Update source_definitions.yaml
Update facebook-marketing.md
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉 New Source: Apple Search Ads [low-code CDK] (#19557)
chore(source-apple-search-ads): add low-code boilerplate code
feat(source-apple-search-ads): add spec and 6 streams
test(source-apple-search-ads): set integration and acceptance tests
fix(source-apple-search-ads): camel case for all properties
docs(source-apple-search-ads): add bootstrap and general docs
docs(source-apple-search-ads): update changelog with right PR number and content
feat(source-apple-search-ads): add lookback window and set to 30d
chore(source-apple-search-ads): remove unused file
test(source-apple-search-ads): complete sample state
docs(source-apple-search-ads): rename documentation url
docs(source-apple-search-ads): remove 90 days limitation
feat(source-apple-search-ads): add exponential backoff
fix(source-apple-search-ads): fix acceptance test execution
chore(source-apple-search-ads): switch time interval to ISO 8601 duration
chore(source-apple-search-ads): update json schemas to the latest version
test(source-apple-search-ads): ignore fields that are not idempotent
test(source-apple-search-ads): align threshold days with lookback window for incremental test
test(source-apple-search-ads): adjust timeout
chore(source-apple-search-ads): add latest CDK changes to boilerplate code
refactor(source-apple-search-ads): use latest manifest for CDK
add icon and source seed
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉 New Source: Aircall [Low code CDK] (#25240)
Initial Commit
fix sample config
change date_from to start_date
optimize retriever method
paginate records
fix formating
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Increase memory_limit for Amplitude (#25282)
Increase memory_limit for Amplitude
Automated Change
Adds 400 errors and reauth suggestion to Airtable doc (#25255)
Adds information about 400 and 401 errors that can be temporarily fixed by reauthing
Update docs/integrations/sources/airtable.md
🐛 Source Jira: Add missing types to issues json schema (#25275)
[FIX] add types
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump source-jira version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump oss version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
CAT tools/acceptance_test_config_migration: Make relatively configurable per migration, add (hacky) ability to run CAT tests locally for multiple connectors (#24377)
Pass argument along, add test that should pass and test that should fail
Add tests with additionalProperties
Set additionalproperties=false when not set|
Parametrize test cases
Make the behavior 'optional'
Fix parametrization for all combinations
Improve CI credentials README and rename param
Update naming to be clearer about columns only
record_has_unexpected_field > record_has_unexpected_column
Automated Change
Hacking the CAT dockerfile and run script to test my changes specifically
First crack at running CAT on all connectors
Write during instead of after all tests
Async-ify it
Add ability to define max concurrency
Write successes
ci_credentials: fix overwriting 'data' before getting nextPageToken
Adjustible num_semaphores, check to make sure it's an airbyte connector first
Automated Change
Make create_issues and create_prs more configurable, add issue for fail_on_extra_columns
Add ability to pass in sources as a list or from a txt file
Add logs to issue, make project nullable
Migrate multiple connectors
Add cli args
use ruamel.yaml to preserve ordering
Separate config loading from config migration
Add ability to pass in lists of sources to test. Sort output by exit codes. Fix max_concurrency flag
Default to testing only beta and GA connectors
Always write test output when available
Revert "Add cli args"
This reverts commit b538a8c.
Remove slash
Don't run on alpha connetors, handle older config style
Don't migrate to new format, preserve quotes and long lines
Automated Change
Update issue, don't run for alpha connectors
Automated Change
Add bypass for extra fields test
Add bypass for extra fields test
Rename run_tests script
Rename module
Update args usage, small changes
Refactor create_issues.py
Clean up run_tests.py
Sort out arg parsers
Pull out get_valid_definitions_from_args
Import definitions module instead of methods
Use config files to provide constants for each migration
Handle FileNotFoundError in create_issues.py, improve logging
Rename to migrations, reference name of folder via utils
Update readmes for migration modules, add script for getting outputs
Use tmp dir, correct path for issue reference
Fix bash script
Fix create command, pull out test results insertion
Update call to update_configuration
add precommit to requirements
Reorder README
README cleanup for test and create issues
README cleanup for create_prs and config_migration
More readmes! Readmes galore
allow_beta
Restore hacky changes to dockerfile and acceptance-test-docker
Handle 'other' release stages
Update readme
Remove TODO, add comments to shell script
format according to gradle
format
Fix formatting
Co-authored-by: marcosmarxm marcosmarxm@users.noreply.github.com Co-authored-by: erohmensing erohmensing@users.noreply.github.com
Dagster: rename catalog to registry (#25254)
rename catalog to registry in metadata service
rename catalog to registry in metadata files
Run generate models
Fix missed renames
Add github personal access token
Run black
Automated Change
Co-authored-by: bnchrch bnchrch@users.noreply.github.com
Source Amplitude: Reduce request_time_range (#25281)
Source Amplitude: Reduce request_time_range
Source Amplitude: Reduce request_time_range
Update docs/integrations/sources/amplitude.md
Co-authored-by: Catherine Noll clnoll@users.noreply.github.com
Update docs/integrations/sources/amplitude.md
auto-bump connector version
Co-authored-by: Serhii Lazebnyi 53845333+lazebnyi@users.noreply.github.com Co-authored-by: Catherine Noll clnoll@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
increase postgres heartbeat from 3600->7200 (#25304)
connectors-ci: fix java connectors testing (#25268)
connectors-ci: include destination-azure-blob-storage gradle project as a destination build dep
fix integration test return value
Source Stripe: add account stream (#25136)
add accounts stream
add docs
lint
fix
fix
fix test
fix tests
fix account schema
fix expected records and schema
remove account from abnormal state
update dockerfile and update docs
Update Dockerfile
Update source_definitions.yaml
Update test_source stream count
auto-bump connector version
Co-authored-by: Haithem Souala haithem.souala@woopit.fr Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
remove session token auth (#25142)
Connector builder: Tutorial (#25153)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
first version of tutorial
review comments
improvements
improvements
review comments
Source Hubspot: update expected records (#25309)
Source Hubspot: update expected records
Update expected records
Bump Airbyte version from 0.44.0 to 0.44.1
🎉 Source Google Ads - fixed schema type for "data_type: ENUM" and "is_repeated: true" to array of strings (#24945)
fixed schema type for "data_type: ENUM" and "is_repeated: true" to array of strings
added SAT for GAQL
updated version
Added unittest for get_json_schema
slight enhancement
fixed empty lines
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
source-amplitude: Only fetch date_time_fields once per request (#25315)
postgres-cdc checkpointing: fix LSN parsing bug + refactor for efficiency (#24582)
Fix LSN parsing from Integer to Long
rebasing
Rebase
Rebase
Other casting
Lock the file only when reading, so the file is free when parsing the object. Increased from 1 to 166 checkpoints, and from skipping hundreds of checkpoints to never skip a state.
Update load function documentation
bump mysql and mssql
cdc: refactor to remove debezium dependency from connector packages
use gradle's shared dependency
more refactoring
upgrade docker version
resolve master merge conflicts
Automated Change
minor changes
resolve merge conflicts
avoid deserializing multiple times
simplify
enable checkpointing for Postgres
more improvements
enable assertions
changelog + bump version
auto-bump connector version
auto-bump connector version
manual bump
Co-authored-by: subodh subodh1810@gmail.com Co-authored-by: subodh1810 subodh1810@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
use actual release tag (#25311)
Source Twilio: update expected records (#25320)
Builder documentation: Pagination documentation (#25261)
add offset increment and page increment docs
add cursor pagination docs
add full example of requests submitted by connector
improve examples and explanation of page/cursor increment
improve descriptions
add section about accessing last record for cursor pagination
add recommendation about page size
add links to low-code and python CDKs
🐛 Destination AWS Datalake: fix airbyte type in complex data types (#24642)
[UPD] prefer airbyte type in complex data types
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] only for integers
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [FIX] changelog table
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] cast objects that can have additional props to string
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [ADD] tests
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Sajarin sajarindider@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Dagger: Return true on valid exit in Dagger Commands (#25333)
Return true on valid exit
Update return type to bool
update to new version of dagger cli (#25340)
🎉Source Mailjet-Mail: Add
listrecipientstream (#24689)add listrecipient stream
update schemas to match source api
bump dockerfile version
update changelog
auto-bump connector version
Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
24153 Destination Databricks: Fix support for external tables on S3 (#24657)
24153 Destination Databricks: Update s3 source external
24153 Destination Databricks: Fixed databricks.md formatting
24153 Destination Databricks: Update the version
auto-bump connector version
Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add version info to publish comment (#25343)
Update managing-airbyte-cloud.md (#25341)
Update managing-airbyte-cloud.md
Update managing-airbyte-cloud.md
Update managing-airbyte-cloud.md
Destination BigQuery + BigQuery denormalized: add configurable file buffer count (#25287)
add configurable file buffer count for dest bq + bq denorm
update changelog
changelog nits
auto-bump connector version
manually bump connector version for bq denorm
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
[JDBC sources] : Log database indexes per stream (#25345)
[JDBC sources] : Log database indexes per stream
Docker & doc bump
auto-bump connector version
auto-bump connector version
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
update link (#25349)
connectors-ci: build multi platform connector images (#25286)
🎉 New Destination: Cumul.io (#23153)
Included Cumul.io Destination connector
add gradle file
add dockerfile and format files
Refactor DestinationCumulio class based on review
This commit refactors the DestinationCumulio class by removing some unused dummy variables and commented code that used to push data in the check method. Next to that I fixed a potential bug as pointed out by @grishick who reviewed this code (thanks for that!). Previously, the state message could have been yielded before any buffers were flushed, which could result in incorrectly indicating a synchronization has been completed while data could still be inside the buffer. Now, all buffers will be flushed before yielding that message.
Chore: add Cumul.io logo svg to airbyte-config/init/src/main/resources/icons/
Update two unit tests that were failing due to fixing bug in a previous commit
fix tests
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
Source SFTP-Bulk: Support custom CSV separators (#19224)
manage different separators
tests
update tests
add optional parameter for separator
add default separator
typo
add trailing newline
fix spec.json order
bump dockerfile version
update changelog
auto-bump connector version
retrigger checks
retrigger checks
Co-authored-by: Sunny Hashmi 6833405+sh4sh@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Amplitude: refactor Events stream (#25317)
Source Amplitude: Refactor Events Stream, based on Python CDK
Source Amplitude: bump docker version
Source Amplitude: Fix add get_updated_state
Revert "Increase memory_limit for Amplitude (#25282)"
This reverts commit 06354ae.
Source Amplitude: Fix backport compatibility with new config
Update source_definitions.yaml
Source Amplitude: update catalogs
Co-authored-by: artem1205 artem1205@users.noreply.github.com
low-code: Do not apply transforms on AirbyteLogMessages and AirbyteTraceMessages (#25290)
Check the input type before applying transformations
format
remove debug prints
connectors-ci: use shared pip cache instead of locked (#25370)
fix typo in GHA workflow (#25372)
Dagster registry generation and persist (#25260)
Define legacy and latest registry at same time
Fix file persist
Get persist to work
Expand generate models to all models
Add new registry types
Fix class name
Get valid registry type definition
Remove uuid hack
Fix ids for json schemas
Resolve issues
Update legacy assets
Add typed legacy registry
Fix icon issue
Regenerate models
Update spec mask to use registry type
Move v1 to v0
Add json sanitized dict helper
Fix tests and format
Ensure we only get latest
Code review comments
fix missing spec error
Move registry code to helper
Destination Databricks: use hive_metastore if catalog not set (#25366)
use hive_metastore if catalog not set
Automated Change
add test case
version bump + changelog
auto-bump connector version
Co-authored-by: edgao edgao@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Salesforce: update fields in Account expected_records (#25385)
Source Faker: update expected records (#25387)
Source Freshsales: update expected records (#25388)
Source Amplitude: update changelog (#25392)
Update Issues Templates (#25365)
update issue templates
add titles and fix multiline text
Automated Change
Update issue-cli.yaml
Update issue-connector.yaml
Update issue-platform.yaml
Update issue-connector.yaml
Docs: Add AWS_SECRET_MANAGER to list of available options for storing secrets in documentation (#25374)
Source Snapchat Marketing: Add missing story metrics to basic stats (#20897)
Add missing story metrics to basic stats
fix expected records and update dockerfile version
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
- Fix Java destination template (#25390)
The currently generated Java destination code doesn't compile. Fix it up.
Connector builder: Incremental sync documentation (#25238)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
review comments
partitioning documentation
incremental sync
improvements
review comments
Conenctor builder: Record processing page (#25267)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
first version of tutorial
review comments
improvements
partitioning documentation
review comments
improvements
record processing doc
review comments
🎉 New Destination: Starburst Galaxy (#25399)
Co-authored-by: Mayank Vadariya 48036907+mayankvadariya@users.noreply.github.com
Add git checkout (#25376)
Destination S3 Glue: Fix precision and scale of decimal type (#25397)
destination-s3-glue: Fix precision and scale of decimal type
The decimal type in Hive will default to a precision of 10 and a scale of 0, meaning that it cannot handle numbers larger than 10 digits. The maximum values for precision and scale are 38 and 38 respectively. This updates the decimal type specified in destination schemas to use the maximum values for the broadest compatibility with numeric values.
If a table is using the default precision of 10 and a number is present which exceeds
that many digits then the query engine will throw a fieldValue cannot be null error
message which is very cryptic.
run seed generator
Automated Change
Update connectors.md
Automated Change
Co-authored-by: Tobias Macey tmacey@mit.edu
small docs fixes (#25377)
Add descriptions and examples to component schema (#25117)
Reference docs for backoff strategies
Docs for most authentication mechansims
Docs for CheckStream
MinMaxDatetime
DeclarativeStream
DefaultErrorHandler
CompositeErrorHandler
update
Update token expiry date description
DPath extractor
Add interpolation_context
HttpResponseFilter
RecordFilter
RecordSelector
DefaultPaginator
CursorPagination
OffsetIncrement
Page Increment
PrimaryKey
HttpRequester
request option and request path
Schemas
Spec
Add field
remove fields
no auth and no pagination
Delete deprecated comment
Missing description
Json Decoder
OAuthConfigSpecification
reorder
add titles, examples, and descriptions for partition routers and datetime based cursor
updates
Update
fix indentation
Automated Commit - Formatting Changes
Update as per feedback
html tag
generate models
Update name
do not use title as class name
Update
Add stream_interval and stream_partition to interpolation_context
Automated Commit - Formatting Changes
fix path
format
Co-authored-by: brianjlai brian.lai@airbyte.io Co-authored-by: girarda girarda@users.noreply.github.com
Fix and document macros and interpolation variables (#25305)
Fix and document macros
cleanup
dots
Add tests and refactor
Update
Add an example
Document variables
Mention now_local is not recommended
🤖 Bump patch version of Airbyte CDK
Source Twilio: update messages stream expected records (#25384)
🎉 Source Stripe: add SetupIntents stream (#23963)
add SetupIntents schema
add setup intents
add test & docs
update docs
add additionalProperties
update date
fix schema
update schema and expected records
fix sub schedule schema
update expected records for sub schedule
update expected records
update source def seed
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: Mal Hancock mallory@archangelic.space Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
connectors-ci: document the CLI (#25391)
connectors-ci: package normalization inside destinations images supporting it (#25359)
Update oss_catalog.json
fix conflicts
fix conflict oss
fix conflict oss
fix conflict oss
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
fix reviews expected records, add EOF
fix email_analytics schema
fix expected records
Update sample_config.json
Update source_definitions.yaml
auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Visnu Pitiyanuvath visnup@gmail.com Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Augustin augustin@airbyte.io Co-authored-by: Alexandre Girard alexandre@airbyte.io Co-authored-by: girarda girarda@users.noreply.github.com Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Artem Inzhyyants 36314070+artem1205@users.noreply.github.com Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com Co-authored-by: Arsen Losenko 20901439+arsenlosenko@users.noreply.github.com Co-authored-by: arsenlosenko arsenlosenko@users.noreply.github.com Co-authored-by: Ali Hassan alihassan01264@gmail.com Co-authored-by: Ali Hassan 62539376+Alihassanc5@users.noreply.github.com Co-authored-by: Serhii Lazebnyi 53845333+lazebnyi@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Boluwatife Victor 95125924+BirdboyBolu@users.noreply.github.com Co-authored-by: Rodi Reich Zilberman 867491+rodireich@users.noreply.github.com Co-authored-by: rodireich rodireich@users.noreply.github.com Co-authored-by: Philip Corr PhilipCorr@users.noreply.github.com Co-authored-by: Lake Mossman lake@airbyte.io Co-authored-by: evantahler evantahler@users.noreply.github.com Co-authored-by: Davin Chia davinchia@gmail.com Co-authored-by: Cole Snodgrass cole@airbyte.io Co-authored-by: colesnodgrass colesnodgrass@users.noreply.github.com Co-authored-by: timroes timroes@users.noreply.github.com Co-authored-by: Sajarin sajarindider@gmail.com Co-authored-by: Rachel RIZK rachel.rizk@backmarket.com Co-authored-by: sajarin sajarin@users.noreply.github.com Co-authored-by: Denys Davydov davydov.den18@gmail.com Co-authored-by: Tim Roes tim@airbyte.io Co-authored-by: Ben Church ben@airbyte.io Co-authored-by: Topher Lubaway asimplechris@gmail.com Co-authored-by: Jimmy Ma gosusnp@users.noreply.github.com Co-authored-by: Akash Kulkarni 113392464+akashkulk@users.noreply.github.com Co-authored-by: Ivica Taseski ivica.taseski94@gmail.com Co-authored-by: Prateek Mukhedkar 123108018+prateekmukhedkar@users.noreply.github.com Co-authored-by: Ryan Fu ryan.fu@airbyte.io Co-authored-by: richardlam-quotapath 113922446+richardlam-quotapath@users.noreply.github.com Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: akashkulk akashkulk@users.noreply.github.com Co-authored-by: Joe Reuter joe@airbyte.io Co-authored-by: bnchrch bnchrch@users.noreply.github.com Co-authored-by: Vladimir remar vladimir.remar@gmail.com Co-authored-by: Rachel Rizk 69161287+rach-r@users.noreply.github.com Co-authored-by: Catherine Noll clnoll@users.noreply.github.com Co-authored-by: Juan 80164312+jnr0790@users.noreply.github.com Co-authored-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Ella Rohm-Ensing erohmensing@gmail.com Co-authored-by: erohmensing erohmensing@users.noreply.github.com Co-authored-by: Haithem Souala haithem.souala@woopit.fr Co-authored-by: josephkmh josephkmh@users.noreply.github.com Co-authored-by: midavadim midavadim@yahoo.com Co-authored-by: Sergio Ropero 42538006+sergio-ropero@users.noreply.github.com Co-authored-by: subodh subodh1810@gmail.com Co-authored-by: subodh1810 subodh1810@users.noreply.github.com Co-authored-by: Conor cpdeethree@users.noreply.github.com Co-authored-by: Dominik Wagner dominik.r.mall2@gmail.com Co-authored-by: Yevhen Sukhomud suhomud@gmail.com Co-authored-by: Pedro S. Lopez pedroslopez@me.com Co-authored-by: Riley Brook riley@airbyte.io Co-authored-by: Cynthia Yin cynthia@airbyte.io Co-authored-by: Joost Stessens 77833265+jooststessens@users.noreply.github.com Co-authored-by: Ana Loureiro 37600275+anamargaridarl@users.noreply.github.com Co-authored-by: Edward Gao edward.gao@airbyte.io Co-authored-by: edgao edgao@users.noreply.github.com Co-authored-by: Chris Wu chris@faros.ai Co-authored-by: iberchid 93732608+iberchid@users.noreply.github.com Co-authored-by: Pekka Enberg penberg@iki.fi Co-authored-by: Mayank Vadariya 48036907+mayankvadariya@users.noreply.github.com Co-authored-by: Tobias Macey tmacey@mit.edu Co-authored-by: brianjlai brian.lai@airbyte.io Co-authored-by: JJ Nilbodee pornwipa.nil@gmail.com Co-authored-by: Mal Hancock mallory@archangelic.space
marcosmarxm pushed a commit to natalia-miinto/airbyte that referenced this pull request
24153 Destination Databricks: Update s3 source external
24153 Destination Databricks: Fixed databricks.md formatting
24153 Destination Databricks: Update the version
auto-bump connector version
Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
marcosmarxm added a commit to natalia-miinto/airbyte that referenced this pull request
Initial commit
Add stream top_reviews
Add new stream: privacy_data
add new stream: privacy user, refactor
Add new streams, update docs, update definitions
EOF
Update spec.yaml
fix date_from, optimize manifest, update docs
delete privacy streams, refactor webhooks
add pagination
add streams raw_data, email_analytics
add streams unsubscribers, webhook_events
Fix pagination
revert doc
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
Revert "remove top_reviews and reviews_bottomline, Make common paginator"
This reverts commit 6615288b6d2441d9fdde2d927a5eb77b5d788442.
remove streams top_reviews, reviews_bottomline
make common base and extractor
refactor, Checkpoint for full test passed
refactor base, requester, paginator
remove extra paginator property
Update oss_catalog.json
fix conflicts
fix conflict oss
fix conflict oss
Add stream top_reviews
Add new stream: privacy_data
add new stream: privacy user, refactor
Add new streams, update docs, update definitions
EOF
Source Stripe: expand tiers when syncing plans (airbytehq#19906)
Expand tiers when syncing plans
Update Plans stream schema
Re-bump
Bump
Remove string as a plan schema
bump
Update changelog
Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
airbyte-ci: improve metadata validation perfs (airbytehq#25107)
metadata service: upload metadata files to gcs (airbytehq#25115)
metadata: cleanup workflows (airbytehq#25105)
Source Stripe: generate seed source file (airbytehq#25113)
Expand tiers when syncing plans
Update Plans stream schema
Re-bump
Bump
Remove string as a plan schema
bump
Update changelog
update source spec
Automated Change
Co-authored-by: Visnu Pitiyanuvath visnup@gmail.com Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com
Connector builder: set pages and slices limits (airbytehq#25121)
Set limits
refactor and add unit tests
Update as per comments
🤖 Bump patch version of Airbyte CDK
Bump Airbyte version from 0.43.1 to 0.43.2
source-postgres works with bigger datasets (airbytehq#25138)
create metadata files for all connectors (airbytehq#24964)
fix date_from, optimize manifest, update docs
Update spec.yaml
🚨🚨 Source Amplitude: migrate to YAML (airbytehq#22362)
Source Amplitude: migrate to YAML
Source Amplitude: enable high strictness level
Source Amplitude: update docs
Source Amplitude: update cursor field + expected records
Source Amplitude: logging refactor
Source Amplitude: refactor extractors
Source Amplitude: refactor extractors
Source Amplitude: migrate to beta low-code
Source Amplitude: add docstring
Source Amplitude: fix unit tests
Source Amplitude: update expected records
Source Amplitude: fix response filter action
Source Amplitude: rename typo
Source Amplitude: fix migration
Source Amplitude: update docs
Source Amplitude: fix migration
Source Amplitude: update expected recrods
Source Amplitude: bump manifest version
Automated Change
Source Amplitude: Update http -> https
Source Amplitude: tests fix + pin cdk version
auto-bump connector version
Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Support: update expected records (airbytehq#25058)
Source Zendesk Support: skip stream
Add expected records for tags stream
Automated Change
Co-authored-by: arsenlosenko arsenlosenko@users.noreply.github.com
Source Surveymonkey: Fix spec.json required fields and update schema for surveys and survey_responses (airbytehq#25080)
Fixes required fields on SurveyMonkey UI
fix integration test schema update for surveys and survey response
fix expected records
update dockerfile and docs
auto-bump connector version
Co-authored-by: Ali Hassan alihassan01264@gmail.com Co-authored-by: Ali Hassan 62539376+Alihassanc5@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add additional destination-bigquery datacenters (airbytehq#25122)
Add additional bigquery datacenters
bump dockerfile
revert source BQ bump
docs
builds depend on normalization now...
no -base
auto-bump connector version
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Bump versions in atelier dockerfile (airbytehq#25125)
Bump versions in atelier dockerfile
fix new version
temporarily skip build and publish
reset
skip some steps
Revert "skip some steps"
This reverts commit e034d10.
Source: Amazon Ads - Add pk validator for reports when expected pk is not returned (airbytehq#25146)
Add pk validator for reports when expected pk is not returned
Update PR number
Fix formating
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉Source Twilio: New Stream Conversation Participants (airbytehq#23995)
Source Twilio: New Stream Conversation Participants
fix: format schema and fix to incremental stream
refactor: Add newline
fix: remove unwanted file
fix: schema fixes
fix: Avoid acceptance test bypass for full testing
fix: Fixed incremental behaviour, Updated docs, Updated VersionTag
fix: Fixed incremental behaviour, Updated docs, Updated VersionTag
fix: Fixed syntax, Updated docs
fix: schema, update docs
fix: Changed parent stream, expected record fixes, connector version bump
chore: EOL fix
fix expected records for conversations participant
Update Twilio version to 0.5.0
update twilio version to 0.5.0
remove conversation_participants from state file
add lines and add eof
update twilio version to 0.5.0 in source spec
removed conversation_participants from abnormal state
remove extra lines
update doc connector version
update json schema draft version
fix doc conflict
add expected records
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Iterable: update expected records (airbytehq#25144)
Source Facebook Pages: update insight metrics request params (airbytehq#25143)
Source Facebook Pages: update insight metrics request params
Update Dockerfile and changelog
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
fix strict-encrypt metadata dockerRepository (airbytehq#25159)
Source Slack: update expected records (airbytehq#25150)
dagger: upgrade to 0.5.0 (airbytehq#25171)
dagger: upgrade to 0.5.0
clean class ConnectorPackageInstall(Step):
Fixed missing punctuations (airbytehq#25151)
Update README.md
I gave a better description to the Airbyte Cloud section, converted a few passive words to active, and added a missing comma.
- Update README.md
I noticed several punctuation marks were missing, so I fixed them. I also want to use this opportunity to draw get the attention of the docs team. I would love to join Airbyte as a technical writer and be a part of the team. I am more than willing to handle these tasks, even if the organization does not have the current budget to onboard a new technical writer.
Also, I filled the form to participate in the Airbyte Writer Program. I hope the team reviews my topic and allow me contribute to the data insight and tutorial articles. Cheers!
- Update README.md
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
refactor source harness and GitHub action to add source-mysql (airbytehq#25036)
refactor to add source-mysql
refactor to add source-mysql
refactor to add source-mysql
test
Automated Change
fix if condition
try multiline
new db
add mysql catalogs
Co-authored-by: rodireich rodireich@users.noreply.github.com
- change google analytics dockerfile order to pip install source (airbytehq#23082)
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
Always show client cert and key (airbytehq#25131)
make CA cert and key always show
make same change for mysql
update readmes
update strict-encrypt version and expected spec
fix expected spec for mysql strict encrypt
auto-bump connector version
manually bump seed for postgres and mysql
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🐛 Source File: fix column data types for numerical values (airbytehq#23759)
Fix column data types for numerical values
Update unit tests
Update Dockerfile version
update dockerfile version for source-file-secure
Update source-file in seed file
Update doc
Update source_specs.yaml
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
connectors.mdgains Github labels for connector issues (airbytehq#25182)connectors.mdgains github labelAutomated Commit - Formatting Changes
comment
Automated Commit - Formatting Changes
Automated Change
Co-authored-by: evantahler evantahler@users.noreply.github.com
- Housekeeping: Remove unused workflows. (airbytehq#25191)
This is no longer used with Move's move to Zenhub.
increase sentry heartbeat - 43200->64800 (airbytehq#25185)
increase sentry heartbeat - 43200->64800
Automated Change
Co-authored-by: colesnodgrass colesnodgrass@users.noreply.github.com
Bump Airbyte version from 0.43.2 to 0.44.0
dagger CI: auth to dockerhub (airbytehq#25196)
auth to dockerhub sometime
map docker hub username and password to env var
Revert "auth to dockerhub sometime"
This reverts commit d82bf88.
- Revert "Revert "auth to dockerhub sometime""
This reverts commit bc9bc61.
login to dockerhub
login to dockerhub
revert changes
airbyte-ci: orchestrate the upload of modified metadata.yaml files to GCS (airbytehq#25044)
Source Hubspot: update expected records (airbytehq#25152)
Source Hubspot: update expected records
Update expected records
Source Github: update expected records (airbytehq#25148)
Source Github: update expected records
Update expected records
Move teams expected records to previous place
Source Pipedrive: update expected records (airbytehq#25199)
Source Pipedrive: update expected records
Update expected records
Removed expected record that is not needed
Source Zendesk Support: update expected records (airbytehq#25206)
Source Twilio: update expected records (airbytehq#25200)
connector builder: Emit message at start of slice (airbytehq#25180)
Move condition for yielding the slice message to an overwritable method
Automated Commit - Formatting Changes
yield the slice log messages
same for incremental
refactor
Revert "refactor"
This reverts commit c594365.
move flag from factory to source
set the flag
remove debug print
halfmock
clean up
Add a test for a single page
Add another test
Pass the flag
rename
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump patch version of Airbyte CDK
CI checks for airbytehq#23868 (airbytehq#24717)
feat(source-bing-ads): add dimensional columns for AccountPerformanceReport
feat(source-bing-ads): add dimensional columns for AdGroupPerformanceReport
feat(source-bing-ads): add dimensional columns for AdPerformanceReport
feat(source-bing-ads): add dimensional columns for KeywordPerformanceReport
feat(source-bing-ads): add dimensional columns for CampaignPerformanceReport
docs(source-bing-ads): update changelog
chore(source-bing-ads): bump version in Dockerfile
test(source-bing-ads): allow extra fields for basic read
Revert "test(source-bing-ads): allow extra fields for basic read"
This reverts commit 961f0d6.
test(source-bing-ads): ignore community contributed fields
fix: updated expected_records of report streams
auto-bump connector version
Automated Change
Co-authored-by: Rachel RIZK rachel.rizk@backmarket.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com Co-authored-by: sajarin sajarin@users.noreply.github.com
Source Facebook Marketing: fix data retention period validation (airbytehq#25204)
airbytehq#1678 source Facebook Marketing: fix data retention period validation
airbytehq#1678 source fb marketing: add validation for the date range in the future
airbytehq#1678 source fb marketing: another fix
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add OSS getting started redirect (airbytehq#25213)
Add is_optional flag to connector tests (airbytehq#25211)
Run doc deploys on docosaurus changes (airbytehq#25215)
Co-authored-by: Topher Lubaway asimplechris@gmail.com
Fix manifest_declarative_source + add unit tests (airbytehq#25217)
Fix + unit test
Add a test with pagination
Add a test with partition router
Make sure _fetch_next_page is called with the right arguments
Automated Commit - Formatting Changes
pagination with partitions
refactor
clean up
format
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump patch version of Airbyte CDK
Document the config for docker workspace cleanup (airbytehq#25210)
delete privacy streams, refactor webhooks
add pagination
add streams raw_data, email_analytics
add streams unsubscribers, webhook_events
Fix pagination
revert doc
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
Revert "remove top_reviews and reviews_bottomline, Make common paginator"
This reverts commit 6615288b6d2441d9fdde2d927a5eb77b5d788442.
remove streams top_reviews, reviews_bottomline
make common base and extractor
refactor, Checkpoint for full test passed
refactor base, requester, paginator
remove extra paginator property
Cleanup some logging in connectors (airbytehq#25227)
🎉 New Source: Teradata (airbytehq#24221)
add teradata template
add teradata environment client
add teradata source connector
add docs
fix lowercase letter
fix compilation error & config test
fix timestamp
generate seed
Co-authored-by: Prateek Mukhedkar 123108018+prateekmukhedkar@users.noreply.github.com
Add metadata gcs sensor to Dagster (airbytehq#25055)
Add solution skeleton
Add metadata blob resource
Wire up sensor
Fix report
Materialize metadata definitions from folder
Move metadata to own file
Run black
Add PR comments
Remove dockerhub from action
🎉Source Stripe: New Stream - Subscription Schedule (airbytehq#23624)
bump connector version
Source stripe: New Stream Subscription Schedule
fix: schema fixes, updated docs, updated versionTag
fix: test
Update stripe.md
fix: Add empty stream
schema fix, version bump, source update
unwanted line fix
Resolve comments
fix schemas
add expected records and update schema
🎉Source Twilio: New Stream Conversation Messages (airbytehq#22875)
bump connector version
Source Twilio: New Stream Conversation Messages
fix: Adding correction to syntax
fix: schema errors, incremental stream fix
fix: schema, updated docs, dockerVersion, removed unwanted files
Update twilio.md
fix schema, add expected records and update stream
remove extar empty line
format files
fix schema for conversation messages
fix conversation participants schema
remove conversation participants from incr ctg
remove unused logging
update schema
Update source_definitions.yaml
Update jira.md
Update expected_records.jsonl
Update twilio.md
Update twilio.md
Update twilio.md
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
remove json schema from integration test
add schema json
Update jira.md
Update stripe.md
Update source_definitions.yaml
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add compatibility guide to sidebar (airbytehq#25224)
Shuffled
destination-jdbcintobasesfolder (airbytehq#25120)Testing changes for reshuffle
Resolves dependency/race-condition with base classes and docker creation
Removes dockerfile
Updates java connectors gradle dependencies
Run ./gradlew :airbyte-config:init:processResources (airbytehq#25250)
add note on security self-check to security.md (airbytehq#25235)
Source Copper: Add opportunities to streams (airbytehq#24824)
Add opportunities to copper streams
Fix version
update schema
update versions and fix schema formatting
auto-bump connector version
Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
low-code: Clean up SessionTokenAuthenticator interface (airbytehq#25086)
Username and session token are optional fields
update
Add titles, descriptions, and examples
Automated Commit - Formatting Changes
fix a small typo
Co-authored-by: girarda girarda@users.noreply.github.com
🤖 Bump minor version of Airbyte CDK
Add logging functionality to understand Xmin wraparound (airbytehq#25220)
Add logging functionality
- Add logging for DB version (all JDBC sources)
- Add logging for Xmin values
Automated Change
Renaming + comments
More comments
Bump docker version + docs
auto-bump connector version
Update failed source_specs.yaml
Co-authored-by: akashkulk akashkulk@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
metadata: create specs secret mask (airbytehq#25201)
Connector builder: Authentication documentation (airbytehq#25112)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
review comments
review comments
Connector builder: Partitioning documentation (airbytehq#25209)
partitioning documentation
review comments
Source S3: Add start date filter for files (airbytehq#25010)
Source S3: Add start date filter for files
Source S3: add docs
Source S3: add unittest
Source S3: add unittest
Source S3: add unittest
Source S3: Fix spec test
Source S3: bump version
Source S3: fix tests
Source S3: fix description
auto-bump connector version
Source S3: refactor start_date filtering
Source S3: update setup
Source S3: serialize state for cache
Source S3: refactor skip file filter
Source S3: bump version + update docs
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Chat: update expected records (airbytehq#25271)
Source Sentry: update expected records (airbytehq#25270)
Source S3: handle block size related errors (airbytehq#25067)
Source S3: handle pyarrow block size errors
Source S3: bump version
Automated Change
Source S3: fix null field check
Revert "Automated Change"
This reverts commit dc707f7.
Automated Change
Source S3: bump version + update docs
auto-bump connector version
Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Zendesk Support: update expected records (airbytehq#25272)
Add Dagster Deployment Action skeleton (airbytehq#24929)
Add code skeleton
Remove pr deployments
remove connectors change
Add action git ignore
Update YAML to deploy properly
Add newline at eof
Setup python-executable deploy
Add temporary script
Update readme
Remove github workflow
Add dagger pipeline for deploy
Add github action
Update readme
Remove comment
Move script into dagger pipeline
Add run test helper and skip test logic
Have the dagster orchestrator properly generate the setup.py file
Remove deploy script
Address PR comments
Update lock
Automated Commit - Formatting Changes
Co-authored-by: bnchrch bnchrch@users.noreply.github.com
🎉 Source Facebook Marketing: Increase pause interval time after reach MIN_RATE or MAX_RATE. (airbytehq#22990)
Change min rate and increase time in pauses
Update Dockerfile
Update source_definitions.yaml
Update facebook-marketing.md
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉 New Source: Apple Search Ads [low-code CDK] (airbytehq#19557)
chore(source-apple-search-ads): add low-code boilerplate code
feat(source-apple-search-ads): add spec and 6 streams
test(source-apple-search-ads): set integration and acceptance tests
fix(source-apple-search-ads): camel case for all properties
docs(source-apple-search-ads): add bootstrap and general docs
docs(source-apple-search-ads): update changelog with right PR number and content
feat(source-apple-search-ads): add lookback window and set to 30d
chore(source-apple-search-ads): remove unused file
test(source-apple-search-ads): complete sample state
docs(source-apple-search-ads): rename documentation url
docs(source-apple-search-ads): remove 90 days limitation
feat(source-apple-search-ads): add exponential backoff
fix(source-apple-search-ads): fix acceptance test execution
chore(source-apple-search-ads): switch time interval to ISO 8601 duration
chore(source-apple-search-ads): update json schemas to the latest version
test(source-apple-search-ads): ignore fields that are not idempotent
test(source-apple-search-ads): align threshold days with lookback window for incremental test
test(source-apple-search-ads): adjust timeout
chore(source-apple-search-ads): add latest CDK changes to boilerplate code
refactor(source-apple-search-ads): use latest manifest for CDK
add icon and source seed
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
🎉 New Source: Aircall [Low code CDK] (airbytehq#25240)
Initial Commit
fix sample config
change date_from to start_date
optimize retriever method
paginate records
fix formating
auto-bump connector version
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Increase memory_limit for Amplitude (airbytehq#25282)
Increase memory_limit for Amplitude
Automated Change
Adds 400 errors and reauth suggestion to Airtable doc (airbytehq#25255)
Adds information about 400 and 401 errors that can be temporarily fixed by reauthing
Update docs/integrations/sources/airtable.md
🐛 Source Jira: Add missing types to issues json schema (airbytehq#25275)
[FIX] add types
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump source-jira version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump oss version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
CAT tools/acceptance_test_config_migration: Make relatively configurable per migration, add (hacky) ability to run CAT tests locally for multiple connectors (airbytehq#24377)
Pass argument along, add test that should pass and test that should fail
Add tests with additionalProperties
Set additionalproperties=false when not set|
Parametrize test cases
Make the behavior 'optional'
Fix parametrization for all combinations
Improve CI credentials README and rename param
Update naming to be clearer about columns only
record_has_unexpected_field > record_has_unexpected_column
Automated Change
Hacking the CAT dockerfile and run script to test my changes specifically
First crack at running CAT on all connectors
Write during instead of after all tests
Async-ify it
Add ability to define max concurrency
Write successes
ci_credentials: fix overwriting 'data' before getting nextPageToken
Adjustible num_semaphores, check to make sure it's an airbyte connector first
Automated Change
Make create_issues and create_prs more configurable, add issue for fail_on_extra_columns
Add ability to pass in sources as a list or from a txt file
Add logs to issue, make project nullable
Migrate multiple connectors
Add cli args
use ruamel.yaml to preserve ordering
Separate config loading from config migration
Add ability to pass in lists of sources to test. Sort output by exit codes. Fix max_concurrency flag
Default to testing only beta and GA connectors
Always write test output when available
Revert "Add cli args"
This reverts commit b538a8c.
Remove slash
Don't run on alpha connetors, handle older config style
Don't migrate to new format, preserve quotes and long lines
Automated Change
Update issue, don't run for alpha connectors
Automated Change
Add bypass for extra fields test
Add bypass for extra fields test
Rename run_tests script
Rename module
Update args usage, small changes
Refactor create_issues.py
Clean up run_tests.py
Sort out arg parsers
Pull out get_valid_definitions_from_args
Import definitions module instead of methods
Use config files to provide constants for each migration
Handle FileNotFoundError in create_issues.py, improve logging
Rename to migrations, reference name of folder via utils
Update readmes for migration modules, add script for getting outputs
Use tmp dir, correct path for issue reference
Fix bash script
Fix create command, pull out test results insertion
Update call to update_configuration
add precommit to requirements
Reorder README
README cleanup for test and create issues
README cleanup for create_prs and config_migration
More readmes! Readmes galore
allow_beta
Restore hacky changes to dockerfile and acceptance-test-docker
Handle 'other' release stages
Update readme
Remove TODO, add comments to shell script
format according to gradle
format
Fix formatting
Co-authored-by: marcosmarxm marcosmarxm@users.noreply.github.com Co-authored-by: erohmensing erohmensing@users.noreply.github.com
Dagster: rename catalog to registry (airbytehq#25254)
rename catalog to registry in metadata service
rename catalog to registry in metadata files
Run generate models
Fix missed renames
Add github personal access token
Run black
Automated Change
Co-authored-by: bnchrch bnchrch@users.noreply.github.com
Source Amplitude: Reduce request_time_range (airbytehq#25281)
Source Amplitude: Reduce request_time_range
Source Amplitude: Reduce request_time_range
Update docs/integrations/sources/amplitude.md
Co-authored-by: Catherine Noll clnoll@users.noreply.github.com
Update docs/integrations/sources/amplitude.md
auto-bump connector version
Co-authored-by: Serhii Lazebnyi 53845333+lazebnyi@users.noreply.github.com Co-authored-by: Catherine Noll clnoll@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
increase postgres heartbeat from 3600->7200 (airbytehq#25304)
connectors-ci: fix java connectors testing (airbytehq#25268)
connectors-ci: include destination-azure-blob-storage gradle project as a destination build dep
fix integration test return value
Source Stripe: add account stream (airbytehq#25136)
add accounts stream
add docs
lint
fix
fix
fix test
fix tests
fix account schema
fix expected records and schema
remove account from abnormal state
update dockerfile and update docs
Update Dockerfile
Update source_definitions.yaml
Update test_source stream count
auto-bump connector version
Co-authored-by: Haithem Souala haithem.souala@woopit.fr Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
remove session token auth (airbytehq#25142)
Connector builder: Tutorial (airbytehq#25153)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
first version of tutorial
review comments
improvements
improvements
review comments
Source Hubspot: update expected records (airbytehq#25309)
Source Hubspot: update expected records
Update expected records
Bump Airbyte version from 0.44.0 to 0.44.1
🎉 Source Google Ads - fixed schema type for "data_type: ENUM" and "is_repeated: true" to array of strings (airbytehq#24945)
fixed schema type for "data_type: ENUM" and "is_repeated: true" to array of strings
added SAT for GAQL
updated version
Added unittest for get_json_schema
slight enhancement
fixed empty lines
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
source-amplitude: Only fetch date_time_fields once per request (airbytehq#25315)
postgres-cdc checkpointing: fix LSN parsing bug + refactor for efficiency (airbytehq#24582)
Fix LSN parsing from Integer to Long
rebasing
Rebase
Rebase
Other casting
Lock the file only when reading, so the file is free when parsing the object. Increased from 1 to 166 checkpoints, and from skipping hundreds of checkpoints to never skip a state.
Update load function documentation
bump mysql and mssql
cdc: refactor to remove debezium dependency from connector packages
use gradle's shared dependency
more refactoring
upgrade docker version
resolve master merge conflicts
Automated Change
minor changes
resolve merge conflicts
avoid deserializing multiple times
simplify
enable checkpointing for Postgres
more improvements
enable assertions
changelog + bump version
auto-bump connector version
auto-bump connector version
manual bump
Co-authored-by: subodh subodh1810@gmail.com Co-authored-by: subodh1810 subodh1810@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
use actual release tag (airbytehq#25311)
Source Twilio: update expected records (airbytehq#25320)
Builder documentation: Pagination documentation (airbytehq#25261)
add offset increment and page increment docs
add cursor pagination docs
add full example of requests submitted by connector
improve examples and explanation of page/cursor increment
improve descriptions
add section about accessing last record for cursor pagination
add recommendation about page size
add links to low-code and python CDKs
🐛 Destination AWS Datalake: fix airbyte type in complex data types (airbytehq#24642)
[UPD] prefer airbyte type in complex data types
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] bump version
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] only for integers
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [FIX] changelog table
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [UPD] cast objects that can have additional props to string
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- [ADD] tests
Signed-off-by: Henri Blancke blanckehenri@gmail.com
- auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Sajarin sajarindider@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Dagger: Return true on valid exit in Dagger Commands (airbytehq#25333)
Return true on valid exit
Update return type to bool
update to new version of dagger cli (airbytehq#25340)
🎉Source Mailjet-Mail: Add
listrecipientstream (airbytehq#24689)add listrecipient stream
update schemas to match source api
bump dockerfile version
update changelog
auto-bump connector version
Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
24153 Destination Databricks: Fix support for external tables on S3 (airbytehq#24657)
24153 Destination Databricks: Update s3 source external
24153 Destination Databricks: Fixed databricks.md formatting
24153 Destination Databricks: Update the version
auto-bump connector version
Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Add version info to publish comment (airbytehq#25343)
Update managing-airbyte-cloud.md (airbytehq#25341)
Update managing-airbyte-cloud.md
Update managing-airbyte-cloud.md
Update managing-airbyte-cloud.md
Destination BigQuery + BigQuery denormalized: add configurable file buffer count (airbytehq#25287)
add configurable file buffer count for dest bq + bq denorm
update changelog
changelog nits
auto-bump connector version
manually bump connector version for bq denorm
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
[JDBC sources] : Log database indexes per stream (airbytehq#25345)
[JDBC sources] : Log database indexes per stream
Docker & doc bump
auto-bump connector version
auto-bump connector version
auto-bump connector version
Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
update link (airbytehq#25349)
connectors-ci: build multi platform connector images (airbytehq#25286)
🎉 New Destination: Cumul.io (airbytehq#23153)
Included Cumul.io Destination connector
add gradle file
add dockerfile and format files
Refactor DestinationCumulio class based on review
This commit refactors the DestinationCumulio class by removing some unused dummy variables and commented code that used to push data in the check method. Next to that I fixed a potential bug as pointed out by @grishick who reviewed this code (thanks for that!). Previously, the state message could have been yielded before any buffers were flushed, which could result in incorrectly indicating a synchronization has been completed while data could still be inside the buffer. Now, all buffers will be flushed before yielding that message.
Chore: add Cumul.io logo svg to airbyte-config/init/src/main/resources/icons/
Update two unit tests that were failing due to fixing bug in a previous commit
fix tests
Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com
Source SFTP-Bulk: Support custom CSV separators (airbytehq#19224)
manage different separators
tests
update tests
add optional parameter for separator
add default separator
typo
add trailing newline
fix spec.json order
bump dockerfile version
update changelog
auto-bump connector version
retrigger checks
retrigger checks
Co-authored-by: Sunny Hashmi 6833405+sh4sh@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Amplitude: refactor Events stream (airbytehq#25317)
Source Amplitude: Refactor Events Stream, based on Python CDK
Source Amplitude: bump docker version
Source Amplitude: Fix add get_updated_state
Revert "Increase memory_limit for Amplitude (airbytehq#25282)"
This reverts commit 06354ae.
Source Amplitude: Fix backport compatibility with new config
Update source_definitions.yaml
Source Amplitude: update catalogs
Co-authored-by: artem1205 artem1205@users.noreply.github.com
low-code: Do not apply transforms on AirbyteLogMessages and AirbyteTraceMessages (airbytehq#25290)
Check the input type before applying transformations
format
remove debug prints
connectors-ci: use shared pip cache instead of locked (airbytehq#25370)
fix typo in GHA workflow (airbytehq#25372)
Dagster registry generation and persist (airbytehq#25260)
Define legacy and latest registry at same time
Fix file persist
Get persist to work
Expand generate models to all models
Add new registry types
Fix class name
Get valid registry type definition
Remove uuid hack
Fix ids for json schemas
Resolve issues
Update legacy assets
Add typed legacy registry
Fix icon issue
Regenerate models
Update spec mask to use registry type
Move v1 to v0
Add json sanitized dict helper
Fix tests and format
Ensure we only get latest
Code review comments
fix missing spec error
Move registry code to helper
Destination Databricks: use hive_metastore if catalog not set (airbytehq#25366)
use hive_metastore if catalog not set
Automated Change
add test case
version bump + changelog
auto-bump connector version
Co-authored-by: edgao edgao@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
Source Salesforce: update fields in Account expected_records (airbytehq#25385)
Source Faker: update expected records (airbytehq#25387)
Source Freshsales: update expected records (airbytehq#25388)
Source Amplitude: update changelog (airbytehq#25392)
Update Issues Templates (airbytehq#25365)
update issue templates
add titles and fix multiline text
Automated Change
Update issue-cli.yaml
Update issue-connector.yaml
Update issue-platform.yaml
Update issue-connector.yaml
Docs: Add AWS_SECRET_MANAGER to list of available options for storing secrets in documentation (airbytehq#25374)
Source Snapchat Marketing: Add missing story metrics to basic stats (airbytehq#20897)
Add missing story metrics to basic stats
fix expected records and update dockerfile version
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
- Fix Java destination template (airbytehq#25390)
The currently generated Java destination code doesn't compile. Fix it up.
Connector builder: Incremental sync documentation (airbytehq#25238)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
review comments
partitioning documentation
incremental sync
improvements
review comments
Conenctor builder: Record processing page (airbytehq#25267)
authentication documentation
some fixes
fix broken link
remove session token auth
review comments
first version of tutorial
Revert "first version of tutorial"
This reverts commit bbdeb99.
first version of tutorial
review comments
improvements
partitioning documentation
review comments
improvements
record processing doc
review comments
🎉 New Destination: Starburst Galaxy (airbytehq#25399)
Co-authored-by: Mayank Vadariya 48036907+mayankvadariya@users.noreply.github.com
Add git checkout (airbytehq#25376)
Destination S3 Glue: Fix precision and scale of decimal type (airbytehq#25397)
destination-s3-glue: Fix precision and scale of decimal type
The decimal type in Hive will default to a precision of 10 and a scale of 0, meaning that it cannot handle numbers larger than 10 digits. The maximum values for precision and scale are 38 and 38 respectively. This updates the decimal type specified in destination schemas to use the maximum values for the broadest compatibility with numeric values.
If a table is using the default precision of 10 and a number is present which exceeds
that many digits then the query engine will throw a fieldValue cannot be null error
message which is very cryptic.
run seed generator
Automated Change
Update connectors.md
Automated Change
Co-authored-by: Tobias Macey tmacey@mit.edu
small docs fixes (airbytehq#25377)
Add descriptions and examples to component schema (airbytehq#25117)
Reference docs for backoff strategies
Docs for most authentication mechansims
Docs for CheckStream
MinMaxDatetime
DeclarativeStream
DefaultErrorHandler
CompositeErrorHandler
update
Update token expiry date description
DPath extractor
Add interpolation_context
HttpResponseFilter
RecordFilter
RecordSelector
DefaultPaginator
CursorPagination
OffsetIncrement
Page Increment
PrimaryKey
HttpRequester
request option and request path
Schemas
Spec
Add field
remove fields
no auth and no pagination
Delete deprecated comment
Missing description
Json Decoder
OAuthConfigSpecification
reorder
add titles, examples, and descriptions for partition routers and datetime based cursor
updates
Update
fix indentation
Automated Commit - Formatting Changes
Update as per feedback
html tag
generate models
Update name
do not use title as class name
Update
Add stream_interval and stream_partition to interpolation_context
Automated Commit - Formatting Changes
fix path
format
Co-authored-by: brianjlai brian.lai@airbyte.io Co-authored-by: girarda girarda@users.noreply.github.com
Fix and document macros and interpolation variables (airbytehq#25305)
Fix and document macros
cleanup
dots
Add tests and refactor
Update
Add an example
Document variables
Mention now_local is not recommended
🤖 Bump patch version of Airbyte CDK
Source Twilio: update messages stream expected records (airbytehq#25384)
🎉 Source Stripe: add SetupIntents stream (airbytehq#23963)
add SetupIntents schema
add setup intents
add test & docs
update docs
add additionalProperties
update date
fix schema
update schema and expected records
fix sub schedule schema
update expected records for sub schedule
update expected records
update source def seed
auto-bump connector version
Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: Mal Hancock mallory@archangelic.space Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com
connectors-ci: document the CLI (airbytehq#25391)
connectors-ci: package normalization inside destinations images supporting it (airbytehq#25359)
Update oss_catalog.json
fix conflicts
fix conflict oss
fix conflict oss
fix conflict oss
Merge branch 'yotpo' of https://github.com/btkcodedev/airbyte into yotpo
fix reviews expected records, add EOF
fix email_analytics schema
fix expected records
Update sample_config.json
Update source_definitions.yaml
auto-bump connector version
Signed-off-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Visnu Pitiyanuvath visnup@gmail.com Co-authored-by: Sherif A. Nada snadalive@gmail.com Co-authored-by: sh4sh 6833405+sh4sh@users.noreply.github.com Co-authored-by: Marcos Marx marcosmarxm@users.noreply.github.com Co-authored-by: Augustin augustin@airbyte.io Co-authored-by: Alexandre Girard alexandre@airbyte.io Co-authored-by: girarda girarda@users.noreply.github.com Co-authored-by: Evan Tahler evan@airbyte.io Co-authored-by: Artem Inzhyyants 36314070+artem1205@users.noreply.github.com Co-authored-by: artem1205 artem1205@users.noreply.github.com Co-authored-by: Octavia Squidington III octavia-squidington-iii@users.noreply.github.com Co-authored-by: Arsen Losenko 20901439+arsenlosenko@users.noreply.github.com Co-authored-by: arsenlosenko arsenlosenko@users.noreply.github.com Co-authored-by: Ali Hassan alihassan01264@gmail.com Co-authored-by: Ali Hassan 62539376+Alihassanc5@users.noreply.github.com Co-authored-by: Serhii Lazebnyi 53845333+lazebnyi@users.noreply.github.com Co-authored-by: marcosmarxm marcosmarxm@gmail.com Co-authored-by: Boluwatife Victor 95125924+BirdboyBolu@users.noreply.github.com Co-authored-by: Rodi Reich Zilberman 867491+rodireich@users.noreply.github.com Co-authored-by: rodireich rodireich@users.noreply.github.com Co-authored-by: Philip Corr PhilipCorr@users.noreply.github.com Co-authored-by: Lake Mossman lake@airbyte.io Co-authored-by: evantahler evantahler@users.noreply.github.com Co-authored-by: Davin Chia davinchia@gmail.com Co-authored-by: Cole Snodgrass cole@airbyte.io Co-authored-by: colesnodgrass colesnodgrass@users.noreply.github.com Co-authored-by: timroes timroes@users.noreply.github.com Co-authored-by: Sajarin sajarindider@gmail.com Co-authored-by: Rachel RIZK rachel.rizk@backmarket.com Co-authored-by: sajarin sajarin@users.noreply.github.com Co-authored-by: Denys Davydov davydov.den18@gmail.com Co-authored-by: Tim Roes tim@airbyte.io Co-authored-by: Ben Church ben@airbyte.io Co-authored-by: Topher Lubaway asimplechris@gmail.com Co-authored-by: Jimmy Ma gosusnp@users.noreply.github.com Co-authored-by: Akash Kulkarni 113392464+akashkulk@users.noreply.github.com Co-authored-by: Ivica Taseski ivica.taseski94@gmail.com Co-authored-by: Prateek Mukhedkar 123108018+prateekmukhedkar@users.noreply.github.com Co-authored-by: Ryan Fu ryan.fu@airbyte.io Co-authored-by: richardlam-quotapath 113922446+richardlam-quotapath@users.noreply.github.com Co-authored-by: Mal Hancock mhancock@archangelic.space Co-authored-by: akashkulk akashkulk@users.noreply.github.com Co-authored-by: Joe Reuter joe@airbyte.io Co-authored-by: bnchrch bnchrch@users.noreply.github.com Co-authored-by: Vladimir remar vladimir.remar@gmail.com Co-authored-by: Rachel Rizk 69161287+rach-r@users.noreply.github.com Co-authored-by: Catherine Noll clnoll@users.noreply.github.com Co-authored-by: Juan 80164312+jnr0790@users.noreply.github.com Co-authored-by: Henri Blancke blanckehenri@gmail.com Co-authored-by: Ella Rohm-Ensing erohmensing@gmail.com Co-authored-by: erohmensing erohmensing@users.noreply.github.com Co-authored-by: Haithem Souala haithem.souala@woopit.fr Co-authored-by: josephkmh josephkmh@users.noreply.github.com Co-authored-by: midavadim midavadim@yahoo.com Co-authored-by: Sergio Ropero 42538006+sergio-ropero@users.noreply.github.com Co-authored-by: subodh subodh1810@gmail.com Co-authored-by: subodh1810 subodh1810@users.noreply.github.com Co-authored-by: Conor cpdeethree@users.noreply.github.com Co-authored-by: Dominik Wagner dominik.r.mall2@gmail.com Co-authored-by: Yevhen Sukhomud suhomud@gmail.com Co-authored-by: Pedro S. Lopez pedroslopez@me.com Co-authored-by: Riley Brook riley@airbyte.io Co-authored-by: Cynthia Yin cynthia@airbyte.io Co-authored-by: Joost Stessens 77833265+jooststessens@users.noreply.github.com Co-authored-by: Ana Loureiro 37600275+anamargaridarl@users.noreply.github.com Co-authored-by: Edward Gao edward.gao@airbyte.io Co-authored-by: edgao edgao@users.noreply.github.com Co-authored-by: Chris Wu chris@faros.ai Co-authored-by: iberchid 93732608+iberchid@users.noreply.github.com Co-authored-by: Pekka Enberg penberg@iki.fi Co-authored-by: Mayank Vadariya 48036907+mayankvadariya@users.noreply.github.com Co-authored-by: Tobias Macey tmacey@mit.edu Co-authored-by: brianjlai brian.lai@airbyte.io Co-authored-by: JJ Nilbodee pornwipa.nil@gmail.com Co-authored-by: Mal Hancock mallory@archangelic.space