Skip to content
Talend-Training.com by Diamond Training
Menu
  • Home
  • Trainings
  • Preview Video
  • Blog
  • Imprint
Home » Data Integration » How to access Google Drive with Talend

How to access Google Drive with Talend

In this article we’re going to see how to access Google Drive with Talend. We’re going to use this “googleDrive” job with a component tGoogleDriveConnection to establish the connection and tGoogleDriveList as an example to list the contents of a folder.

Table of Contents

  • Necessary Steps
  • Enable the Google Drive API
  • Configure the Consent Screen
  • Create necessary Credentials
  • Create Google Drive Connection
  • Test The Connection
  • Create the Job in Talend
    • Adding the Google Drive Connection Component
    • Adding the tGoogleDriveList Component
    • Configure the tGoogleDriveList Component
    • The Job Run Result
  • Other Components to Access Google Drive with Talend
  • Summary
  • Learn More: Complete Talend Course
  • Video Version of this Article

Necessary Steps

The necessary steps I mentioned here in a Note component, but don’t worry we will go through that step by step.
We will do that together and we will not use any paid products.

Google Drive demo job and steps

Google Drive demo job and steps

This is the complete job. I can not execute that here, because I deleted the client that I used to create this one. But we’re going to create this together from scratch.

I already got a small document open here, which says go first to drive.google.com. Here I’ve got this tab already open. And I created a “talend” folder just for this demonstration, with some files inside it.

So you can also create a folder, upload some files as well. Whatever you want.

Then you go to this page console.cloud.google.com. I just added this language tag

hl=en

here at the end, so it’s showing it in English.

The first thing that you have to do there, after signing into your Google account, is select a project. If you don’t have a project, you have to create a new project here.

create project in Google Cloud Console
create project in Google Cloud Console

Just leave the id and organization as is and click on “Create”. It may take a second for the project to be created.

Enable the Google Drive API

There is the project. I say “Select Project”. Once we select the project, we can go to “APIs and Services” here on the left hand side.

Google Drive API menu
Google Drive API menu

We can click on “Enable APIs and Services”.

enable Google Drive API
enable Google Drive API

We want to enable Google Drive API. So it’s this one here.

Google Drive API
Google Drive API

Click on that and click on “Enable”.

Like the project creation, this may also take a moment.

Configure the Consent Screen

Now before we actually click on “Create Credentials”, we will configure the consent screen.

So click on “Credentials” on the left.

credentials menu
credentials menu

We will now “Configure Consent Screen”.

Configure Consent Screen
Configure Consent Screen

Then say it’s an “External” user type and click “Create”.

Consent Screen
Consent Screen settings

Here we give the app a name. It’s “Talend” and it’s the email that you use with Google. We also have to give developer contact information. Here I’ll provide the same email address again.

Then click “Save and continue” and in the subsequent steps you can just click on “Save and continue”. And when you get to the summary, click on “Back to dashboard”.

This app is ready to go. The next thing is, we have to publish it. So click on “Publish app” and confirm this.

publish app
publish app

Create necessary Credentials

Now we go back here on the left hand side to the small key symbol, where it says “Credentials”.

Select the option here on the top left side to “Create credentials”.

The option we use here is “OAuth Client ID”.

create credentials
create OAuth credentials

Now we’ve got to select an application type. It’s a “Desktop app”. We can call that, for example, “Talend client 1”. Click on create.
Here we can now download the access information.

download OAuth credentials
download OAuth credentials

I prefer to copy this Client ID, because I’ve prepared that already. This is what we’re going to use in a moment in Talend. And the “Client Secret” that we’re going to use. Then you confirm on “Ok”.

Create Google Drive Connection

We can now switch over to Talend and do what’s necessary in Talend. Let me close this job.

Here we go to “Metadata”. There we also have an entry which is called “Google Drive”. We right click on this one and select “New GoogleDrive Connection”.

Give it some name and some application name, for example “talend client 1”, like we did in Google. Plus paste the “Client Secret” and the “Client ID”.

Google Drive connection in Talend
Google Drive connection in Talend

Test The Connection

Now we can click on “Test connection” to see if we can access Google Drive with Talend. Have in mind that this opens a tab in your browser.

You have to select the respective account where you created that. Go to “Advanced” here and say “Open Talend”. Then check this box to give it permission and click “Continue”.

allow access
Google Drive connection in Talend

Now it’s saying I can close this window. And in Talend it’s saying connection was established successfully.

Time to click on “Finish”. I got this “talendDrive” connection here. This is what we’re going to use in a job.

Create the Job in Talend

Right click on “Job Designs” and select “Create Job”. The job name is something “googleDrive”. Then click on “Finish”.

Adding the Google Drive Connection Component

Next we’re going to use this “talendDrive” connection. Drag and drop this to your job and convert it to tGoogleDriveConnection. This one should be connected to a tPreJob component, so it’s making sure that always, whatever else comes in the job, we are first establishing the connection.

Here in order to see the unique name of the component in the interface as well, I will copy this part in the “View” tab and put that here and delete just this. So we will see that we are using tGoogleDriveConnection and it’s called “talendDrive”.

label for tGoogleDriveConnection component
label for tGoogleDriveConnection component

Now we give this a title. “Connect to Google Drive”.

Adding the tGoogleDriveList Component

In the next step I want to read from Google Drive. So take this connection from the repository again, drag and drop that into the job. We convert that to tGoogleDriveList, which is very similar to tFileList component, but with the difference that it gives us directly a data stream with the content of a certain folder in Google Drive.

Configure the tGoogleDriveList Component

We can say that it’s not using in this component for the connection, but the tGoogleDriveConnection component. So in “Connection Component” select this one. And here we have to provide the “Folder Name”. For me this is “talend”.

component tGoogleDriveList settings
component tGoogleDriveList settings

The rest is fine. I just want to look at files in this directory and nothing else.

So I connect this to tLogRow component.

Here I will also do the same change on the “View” tab. So it’s going to show the name of the component itself. Just copy and paste that from the other component. So you see in the interface that you’re using tGoogleDriveList component and it’s called “talendDrive”.

Then we’re going to pass this information with a fixed schema to a tLogRow component. It should output the content in “Table” mode.

The Job Run Result

Now we can run this job to see the respective result on the console.

Google Drive job run result
Google Drive job run result

Don’t get worried. There are some warnings in the component itself. And here we got the names of our files and a lot of other information that we could continue using in our job, to do whatever we want.

Other Components to Access Google Drive with Talend

That’s not all we can do with Google Drive in Talend. If we search in the “Palette” for “GoogleDrive”, you can see that there are a lot of Google Drive components.

Google Drive components in Talend Open Studio
Google Drive components in Talend Open Studio

It’s for example tGoogleDrivePut. We could use to upload a file to Google Drive. To get we use the tGoogleDriveGet component.
We can create, delete, copy and some other things.

Summary

So you have seen:

  1. how to access Google Drive with Talend
  2. what you have to create in Google Console for it,
  3. and what components are available

Learn More: Complete Talend Course

If you not only want to know how to access Google Drive with Talend Open Studio, go to my Udemy Course. This is a comprehensive training for using Talend to build data integration jobs, not only for Data Warehouses.

Video Version of this Article

Thank you for your attention and see you soon!

Read other articles.

Get In Touch
Copyright © 2023 Talend-Training.com by Diamond Training – OnePress theme by FameThemes
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Cookie SettingsAccept All
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT