In this article we’re going to see how to access Google Drive with Talend. We’re going to use this “googleDrive” job with a component tGoogleDriveConnection to establish the connection and tGoogleDriveList as an example to list the contents of a folder.
Necessary Steps
The necessary steps I mentioned here in a Note component, but don’t worry we will go through that step by step.
We will do that together and we will not use any paid products.
This is the complete job. I can not execute that here, because I deleted the client that I used to create this one. But we’re going to create this together from scratch.
I already got a small document open here, which says go first to drive.google.com. Here I’ve got this tab already open. And I created a “talend” folder just for this demonstration, with some files inside it.
So you can also create a folder, upload some files as well. Whatever you want.
Then you go to this page console.cloud.google.com. I just added this language tag
hl=enhere at the end, so it’s showing it in English.
The first thing that you have to do there, after signing into your Google account, is select a project. If you don’t have a project, you have to create a new project here.
Just leave the id and organization as is and click on “Create”. It may take a second for the project to be created.
Enable the Google Drive API
There is the project. I say “Select Project”. Once we select the project, we can go to “APIs and Services” here on the left hand side.
We can click on “Enable APIs and Services”.
We want to enable Google Drive API. So it’s this one here.
Click on that and click on “Enable”.
Like the project creation, this may also take a moment.
Configure the Consent Screen
Now before we actually click on “Create Credentials”, we will configure the consent screen.
So click on “Credentials” on the left.
We will now “Configure Consent Screen”.
Then say it’s an “External” user type and click “Create”.
Here we give the app a name. It’s “Talend” and it’s the email that you use with Google. We also have to give developer contact information. Here I’ll provide the same email address again.
Then click “Save and continue” and in the subsequent steps you can just click on “Save and continue”. And when you get to the summary, click on “Back to dashboard”.
This app is ready to go. The next thing is, we have to publish it. So click on “Publish app” and confirm this.
Create necessary Credentials
Now we go back here on the left hand side to the small key symbol, where it says “Credentials”.
Select the option here on the top left side to “Create credentials”.
The option we use here is “OAuth Client ID”.
Now we’ve got to select an application type. It’s a “Desktop app”. We can call that, for example, “Talend client 1”. Click on create.
Here we can now download the access information.
I prefer to copy this Client ID, because I’ve prepared that already. This is what we’re going to use in a moment in Talend. And the “Client Secret” that we’re going to use. Then you confirm on “Ok”.
Create Google Drive Connection
We can now switch over to Talend and do what’s necessary in Talend. Let me close this job.
Here we go to “Metadata”. There we also have an entry which is called “Google Drive”. We right click on this one and select “New GoogleDrive Connection”.
Give it some name and some application name, for example “talend client 1”, like we did in Google. Plus paste the “Client Secret” and the “Client ID”.
Test The Connection
Now we can click on “Test connection” to see if we can access Google Drive with Talend. Have in mind that this opens a tab in your browser.
You have to select the respective account where you created that. Go to “Advanced” here and say “Open Talend”. Then check this box to give it permission and click “Continue”.
Now it’s saying I can close this window. And in Talend it’s saying connection was established successfully.
Time to click on “Finish”. I got this “talendDrive” connection here. This is what we’re going to use in a job.
Create the Job in Talend
Right click on “Job Designs” and select “Create Job”. The job name is something “googleDrive”. Then click on “Finish”.
Adding the Google Drive Connection Component
Next we’re going to use this “talendDrive” connection. Drag and drop this to your job and convert it to tGoogleDriveConnection. This one should be connected to a tPreJob component, so it’s making sure that always, whatever else comes in the job, we are first establishing the connection.
Here in order to see the unique name of the component in the interface as well, I will copy this part in the “View” tab and put that here and delete just this. So we will see that we are using tGoogleDriveConnection and it’s called “talendDrive”.
Now we give this a title. “Connect to Google Drive”.
Adding the tGoogleDriveList Component
In the next step I want to read from Google Drive. So take this connection from the repository again, drag and drop that into the job. We convert that to tGoogleDriveList, which is very similar to tFileList component, but with the difference that it gives us directly a data stream with the content of a certain folder in Google Drive.
Configure the tGoogleDriveList Component
We can say that it’s not using in this component for the connection, but the tGoogleDriveConnection component. So in “Connection Component” select this one. And here we have to provide the “Folder Name”. For me this is “talend”.
The rest is fine. I just want to look at files in this directory and nothing else.
So I connect this to tLogRow component.
Here I will also do the same change on the “View” tab. So it’s going to show the name of the component itself. Just copy and paste that from the other component. So you see in the interface that you’re using tGoogleDriveList component and it’s called “talendDrive”.
Then we’re going to pass this information with a fixed schema to a tLogRow component. It should output the content in “Table” mode.
The Job Run Result
Now we can run this job to see the respective result on the console.
Don’t get worried. There are some warnings in the component itself. And here we got the names of our files and a lot of other information that we could continue using in our job, to do whatever we want.
Other Components to Access Google Drive with Talend
That’s not all we can do with Google Drive in Talend. If we search in the “Palette” for “GoogleDrive”, you can see that there are a lot of Google Drive components.
It’s for example tGoogleDrivePut. We could use to upload a file to Google Drive. To get we use the tGoogleDriveGet component.
We can create, delete, copy and some other things.
Summary
So you have seen:
- how to access Google Drive with Talend
- what you have to create in Google Console for it,
- and what components are available
Learn More: Complete Talend Course
If you not only want to know how to access Google Drive with Talend Open Studio, go to my Udemy Course. This is a comprehensive training for using Talend to build data integration jobs, not only for Data Warehouses.
Video Version of this Article
Thank you for your attention and see you soon!
Read other articles.