Google Technology
Introduction
The Google technology stack is supported in Hop through a number of plugins. We briefly touch upon them below.
Pipeline Transforms
Check the external plugins for more information on a number of additional Google plugins that can’t be or won’t be included with Apache Hop (Google Sheets Input and Output, Google Analytics)
VFS
Apache VFS Support in Hop allows you to directly read from a multitude of file systems and protocols, including Google:
Google Drive: read and write data directly from and to Google Drive files and folders.
Google Cloud Storage: read and write data directly from and to files and folders in Google Cloud Storage buckets
Beam vs Google Cloud
When executing your pipeline using a Beam runner which is NOT DataFlow, make sure to pass the default Google cloud project ID by running:
gcloud config set project <project-id>
This affects Google Cloud specific APIs like BigQuery, Pub/Sub and others.