-->

dev ,  machine_learning google_cloud_platform, speech_to_text

Just out of curiosity I wanted to play around with Google Cloud Platform. They give $300 free credit for a 12 month trial period so I thought this would be a good chance to try it out.

The APIs I wanted to sample were speech recognition and translation.

Setting Up SDK

I followed the quick start guide which is a step-by-step process so it was quite helpful to get acquainted with the basics.

To be able to follow the instructions I downloaded and installed the GCloud SDK. On Mac it’s quite easy:

curl https://sdk.cloud.google.com | bash
exec -l $SHELL
gcloud init

And once it’s complete it requires you to log in to your account and grant access to SDK:

Testing the API

After the initial setup I tried the sample request and it worked just fine:

The example worked but also raised a few questions in my mind:

  1. Sample uses gs protocol. First off, what does it mean?
  2. Can I use good ol’ http instead of it and point to any audio file publicly accessible?
  3. Can I use MP3 as encoding or does it need to be FLAC?

As learned from this SO thread, gs is used for Google Cloud Storage and “https://storage.googleapis.com” translates to “gs://”.

So the http version of the test file is “https://storage.googleapis.com/cloud-samples-tests/speech/brooklyn.flac”. I was able to verify the file actually exists but when I replaced it with the original value I got this error:

{
    "error": {
        "code": 400,
        "message": "Request contains an invalid argument.",
        "status": "INVALID_ARGUMENT"
    }
}

This also answered my second question. According to the documentation it only supports Google Cloud Storage currently:

uri contains a URI pointing to the audio content. 
Currently, this field must contain a Google Cloud Storage URI 
(of format gs://bucket-name path_to_audio_file).

The answer to my 3rd question wasn’t very promising either. Apparently only the types listed below are supported:

If the authorization token expires, you can generate a new one by using the following commands:

export GOOGLE_APPLICATION_CREDENTIALS="/Path/To/Credentials/Json/File"

gcloud auth application-default print-access-token

So no way of uploading a random MP3 and get text out of it. But I’ll of course try anyway :-)

Test Case: Get lyrics for a Rammstein song and translate

OK, now that I have a free trial at my disposal and have everything setup, let’s create some storage, upload some files and put it to a real test.

Step 01: Get some media

My goal is to extract lyrics of a Rammstein song and translate them to English. For that I chose the song Du Hast. Since I couldn’t find a way to download FLAC version of the song I decided to download the official vide from Rammstein’s YouTube channel.

This is just for experimental purposes and I deleted the video after I’m done testing it so should be fine I guess. To download videos from youtube you can refer to this TechAdvisor article.

I simply used VLC to open the YouTube video. In Window -> Media Information dialog it shows the full path of the raw video file and I copied that path into a browser and downloaded the video.

Step 02: Prepare the media to process

Since all I need is audio I extracted it from video file using VLC. Probably can be done in a number of ways but VLC is quite straightforward to do it:

Click File –> Convert & Stream, drag and drop the video

In the Choose Profile section, select Audio - FLAC.

The important bit here is is that by default VLC converts to stereo audio with 2 channels but Google doesn’t support it which is explained in this documentation:

All encodings support only 1 channel (mono) audio

So make sure to customize it and enter 1 as channel count:

Step 03: Call the API

Now I was ready to call the API with my shiny single-channel FLAC file. I uploaded it to the Google Storage bucket I created, gave public access to it and tried the API.

Apparently, speech:recognize endpoint only supports audio up to a minute. This is the error I got after posting a 03:55 audio.

“Sync input too long. For audio longer than 1 min use LongRunningRecognize with a ‘uri’ parameter.”

The solution is using speech:longrunningrecognize endpoint which only returns a JSON with 1 value: name. This is a unique identifier assigned by Google to the job they created for us.

Once we have this id we can query the result of the process by calling GET operations endpoint.

Fantastic! Some results. It’s utterly disappointing of course as we only got a few words out of it, but still something (I guess!).

Step 04: Compare the results:

Now the following is the actual lyrics of the song:

Du
du hast
du hast mich
du hast mich gefragt
du hast mich gefragt, und ich hab nichts gesagt

Willst du bis der Tod euch scheidet
treu ihr sein für alle Tage

Nein

Willst du bis zum Tod, der scheide
sie lieben auch in schlechten Tagen

Nein

and this is what I got back from Google:

du hast 
du hast recht 
du hast 
du hast mich 
du hast mich 
du du hast 
du hast mich

du hast mich belogen

du hast 
du hast mich blockiert

It missed most of the lyrics. Maybe it was headbanging too hard that it couldn’t catch those parts!

Test Case: Slow German Podcast

Since my idea of translating German industrial metal lyrics on the fly failed miserably I decided to try with cleaner audio where there is no music. Found a nice looking podcast called Slow German. Nice thing about it is that it provides transcripts as well so I can compare the Speech API results with it.

Obtained a random episode from their site and followed the steps above.

First 4 paragraphs of the actual transcript of the podcast is as follows (The full transcript can be found here:

Denk ich an Deutschland in der Nacht, dann bin ich um den Schlaf gebracht.“ Habt Ihr diesen Satz schon einmal gehört? Er wird immer dann zitiert, wenn es Probleme in Deutschland gibt. Der Satz stammt von Heinrich Heine. Er war einer der wichtigsten deutschen Dichter. Aber keine Angst: Auch wenn er am 13. Dezember 1797 geboren wurde, sind seine Texte sehr aktuell und relativ leicht zu lesen. Ihr werdet ihn mögen!

Harry Heine wuchs in einem jüdischen Haushalt auf. Er war 13 Jahre alt, als Napoleon in Düsseldorf einzog. Schon als Schüler begann er, Gedichte zu schreiben. Beruflich sollte er eigentlich im Bankgeschäft arbeiten, aber dafür hatte er kein Talent. Also versuchte er es erst mit einem eigenen Geschäft für Stoffe, das aber bald pleite war. Dann begann er zu studieren. Er probierte es mit Jura und mit Geschichte, besuchte verschiedene Vorlesungen.

Mit 25 Jahren veröffentlichte er erste Gedichte. Es war eine aufregende Zeit für ihn. Er wechselte die Städte und die Universitäten, er beendete sein Jura- Studium und wurde promoviert. Um seine Chancen als Anwalt zu verbessern, ließ er sich protestantisch taufen, er kehrte also dem Judentum den Rücken und wurde Christ. Daher auch der neue Name: Christian Johann Heinrich Heine. Später hat er die Taufe oft bereut.

Wenn Ihr Heines Werke lest werdet Ihr merken, dass sie etwas Besonderes sind. Sie sind oft kritisch, sehr oft aber auch ironisch und humorvoll. Er spielt mit der Sprache. Er kann aber auch sehr böse sein und herablassend über Menschen schreiben. Seine Kritik auch an politischen Ereignissen und die Zensur, mit der er in Deutschland leben musste, führten Heinrich Heine nach Paris. Er wanderte nach Frankreich aus.

And this is the result I got from Google (Trimmed to match the above):

denk ich an Deutschland in der Nacht dann bin ich um den Schlaf gebracht habt ihr diesen Satz schon einmal gehört er wird immer dann zitiert wenn es Probleme in Deutschland gibt der Satz stammt von Heinrich Heine er war einer der wichtigsten deutschen Dichter aber keine Angst auch wenn er am 13. Dezember 1797 geboren wurde sind seine Texte sehr aktuell und relativ leicht zu lesen ihr werdet ihn mögen Harry Heine wuchs in einem jüdischen Haushalt auf er war 13 Jahre alt als Nappo

hier in Düsseldorf einen Zoo schon als Schüler begann er Gedichte zu schreiben beruflich sollte er eigentlich im Bankgeschäft arbeiten aber dafür hatte er kein Talent also versuchte er es erst mit einem eigenen Geschäft für Stoffe das aber bald pleite war dann begann er zu studieren er probierte es mit Jura und mit Geschichte besuchte verschiedene Vorlesungen mit 25 Jahren veröffentlichte er erste Gedichte es war eine aufregende Zeit für ihn er wechselte die Städte und die Universitäten er beendete sein Jurastudium und wurde Promo

auch an politischen Ereignissen und die Zensur mit der er in Deutschland leben musste führten Heinrich Heine nach Paris er wanderte nach Frankreich aus

Comparing the translations

Since I don’t speak German I cannot judge how well it did. Clearly it didn’t capture all the words but I wanted to see if what it returned makes any sense anyway. So I put both in Google Translate and this is how they compare:

Translation of the original transcript:

When I think of Germany at night, I'm about to go to sleep. "Have you ever heard that phrase before? He is always quoted when there are problems in Germany. The sentence is by Heinrich Heine. He was one of the most important German poets. But do not worry: even if he was born on December 13, 1797, his lyrics are very up to date and relatively easy to read. You will like him!

Harry Heine grew up in a Jewish household. He was 13 years old when Napoleon moved in Dusseldorf. Even as a student, he began writing poetry. Professionally, he was supposed to work in banking, but he had no talent for that. So he first tried his own business for fabrics, which was soon broke. Then he began to study. He tried law and history, attended various lectures.

At the age of 25 he published his first poems. It was an exciting time for him. He changed cities and universities, he completed his law studies and received his doctorate. To improve his chances as a lawyer, he was baptized Protestant, so he turned his back on Judaism and became a Christian. Hence the new name: Christian Johann Heinrich Heine. Later he often regretted baptism.

When you read Heine's works, you will find that they are special. They are often critical, but often also ironic and humorous. He plays with the language. But he can also be very angry and condescending to write about people. His criticism also of political events and the censorship with which he had to live in Germany led Heinrich Heine to Paris. He emigrated to France.

Translation of Google’s results:

I think of Germany in the night then I'm about to sleep Did you ever hear this sentence He is always quoted when there are problems in Germany The sentence comes from Heinrich Heine He was one of the most important German poets but do not be afraid he was born on December 13, 1797 his lyrics are very up to date and relatively easy to read you will like him Harry Heine grew up in a Jewish household he was 13 years old as Nappo

Here in Dusseldorf a zoo as a student he began to write poetry professionally he should actually work in the banking business but for that he had no talent so he first tried his own business for fabrics but soon broke and then began to study he tried it with Jura and with history attended various lectures at age 25 he published his first poems it was an exciting time for him he changed the cities and the universities he finished his law studies and became promo

also in political events and the censorship with which he had to live in Germany led Heinrich Heine to Paris he emigrated to France

The translations of the podcast are very close, especially the first part. It missed some sentences and when you read the API output at least you can get a general understanding of what the text is about. It’s not a good read maybe and it’s not good if you’re interested in details but it’s probably good enough

Conclusion

Speech to text can be very useful backed with automated real-time translations. Google Speech API supports real time speech recognition as well so it may be interesting to put Translation API in use as well and develop a tool to get real time translations but that’s for another blog post.

Resources

devops git, powershell

Having lots of projects and assets stored on GitHub I thought it might be a good idea to create periodical backups of my entire GitHub account (all repositories, both public and private). The beauty of it is since Git is open source, this way I can migrate my account to anywhere and even host it on my own server on AWS.

Challenges

With the above goal in mind, I started to outline what’s necessary to achieve this task:

  1. Automate calling GitHub API to get all repos including private ones. (Of course one should be aware of GitHub API rate limits which is currently 5000 requests per hour. If you use up all your allowance with your scripts you may not be able to use it yourself. Good thing is they are returning how many calls are left before you exceed your quota in x-ratelimit-remaining HTTP header in their responses.)
  2. Pull all the latest versions for all branches. Overwrite local versions in cases of conflict.
  3. Find a way to easily transfer a git repository (A compressed single file version rather than individual files) if transferring to another medium is required (such as an S3 bucket)

With these challenges ahead, I first started looking into getting the repos from GitHub:

Consuming GitHub API via PowerShell

First, I shopped around for existing libraries for this task (such as PowerShellForGitHub by Microsoft but it didn’t work for me. Basically I couldn’t even manage the samples on their Wiki. It kept giving cmdlet not found error so I gave up.)

Found a nice video on Channel 9 about consuming REST APIs via PowerShell which uses GitHub API as a case study. It was perfect for me as my goal was to use GitHub API anyway. And since this is a generic approach to consume APIs it can come handy in the future as well. It’s quite easy using basic authentication.

Authorization

First step, is to create a Personal Access Token with repo scope. (Make sure to copy the value before you close the page, there is no way to retrieve it afterwards.)

After the access token has been obtained, I had to generate authorization header as shown in the Channel 9 video:

$token = '<YOUR GITHUB ACCOUNT NAME>:<PERSONAL ACCESS TOKEN>'
$base64Token = [System.Convert]::ToBase64String([char[]]$token)
$headers = @{
    Authorization = 'Basic {0}' -f $base64Token
};

$response = Invoke-RestMethod -Headers $headers -Uri https://api.github.com/user/repos

This way I was able to get the repositories including the private ones but by default it returns 30 records on a page so I had to traverse over the pages .

Handling pagination

GitHub sends the next and the last page URLs in link header:

<https://api.github.com/user/repos?page=2>; rel="next", <https://api.github.com/user/repos?page=3>; rel="last"

The challenge here is that looks like Invoke-RestMethod response doesn’t allow to access headers which is a huge bummer as there are useful info in the headers as shown in the screenshot:

GitHub response headers in Postman

At this point, I wanted to use PSGitHub mentioned in the video but as of this writing it doesn’t support getting all repositories. In fact in a note it says “We need to figure out how to handle data pagination” which made me think we are on the same page here (no pun intended!)

GitHub supports a page size parameter (e.g. per_page=50) but the documentation says the maximum value is 100. Although it is tempting to use that one as that would bring all my repos and leave some room for the future ones as well I wanted to go with a more permanent solution. So I decided to request more pages as longs as there are objects returning like this

$page = 1

Do
{
    $response = Invoke-RestMethod -Headers $headers -Uri "https://api.github.com/user/repos?page=$page"
    
    foreach ($obj in $response)
    {
        Write-Host ($obj.id)
    }
    
    $page = $page + 1
}
While ($response.Count -gt 0)

Now in the foreach loop of course I have to do something with the repo information instead of just printing the id.

Cloning / pulling repositories

At this point I was able to get all my repositories. GitHub API only handles account information so now I needed to able to run actual git commands to get my code.

First I had installed PowerShell on Mac which is quite simple as specified in the documentation:

brew tap caskroom/cask
brew cask install powershell

With Git already installed on my machine, all is left was using Git commands to clone or update repo on PowerShell terminal such as:

git fetch --all
git reset --hard origin/master

Since this is just going to be a backup copy I don’t want to deal with merge conflicts and just overwriting everything local.

Another approach could be deleting the old repo and cloning it from scratch but I think this would be a bit wasteful to do it everytime for each and every repository.

Putting it all together

Now that I have all the bits and pieces I have glue them together in a meaningful script than can be scheduled and here it is:

Conclusion and Future Improvements

This version accomplishes the basic task of backing up an entire GitHub account but it can be improved in a few ways. Maybe I can post a follow up article including those improvements. A few ideas come to mind are:

  • Get Gists (private and public) as well.
  • Add option to exclude repos by name or by type (i.e. get only private ones or get all except repo123)
  • Add an option to export them to a “non-git” medium such as an S3 bucket using git bundle (which turns out to be a great tool to pack everything in a repository in a single file)
  • Create a Docker image that contains all the necessary software (Git, PowerShell, backup script etc) so that it can be distributed without any setup requirements.

Resources

aws ,  devops s3, mysql, powershell

I have an application that uses MySQL database. Because of cost concerns it’s running on an EC2 instance instead of RDS. As it’s not a managed environment, the burden of backing up my data falls on me. This is a small step by step guide that details how I’m backing up my MySQL database to AWS S3 with PowerShell

PART 01 - AWS SETUP

  1. Create a bucket named (e.g. “application-backups”) on AWS S3 using AWS Management Console.
  2. Create a new IAM user (e.g “upload-backup-to-s3”)
  3. Create a new policy using management console. The policy will only give enough permissions to put objects into a single S3 bucket:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::xxxxxxx"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": [
                "arn:aws:s3:::xxxxxxx/*"
            ]
        }
    ]
}

In S3 bucket properties copy ARN and replace the x’s above with that.

  1. Customize S3 bucket LifeCycle settings to determine how long you want the old logs retain in your bucket. In my case I set it to expire at 21 days which I think it’s large enough window for relevant database backups. I probably wouldn’t restore anything older than 21 days anyway.

  2. [Optional - For email notifications] Create an SES user by clicking SES -> SMTP Settings -> Create My SMTP Credentials

Make sure you don’t make the mistake I made which was creating an IAM user with a policy that can send emails. In the SMTP settings page there’s a note right below the button:

Your SMTP user name and password are not the same as your AWS access key ID and secret access key. Do not attempt to use your AWS credentials to authenticate yourself against the SMTP endpoint.

When you click the button it creates an IAM user basically for the secret key is 44 bytes whereas the IAM user I created had a secret key of 40 characters. Anyway, bottom line is in order to be able to send emails via SES create the user as described above and all should be fine.

PART 02 - POWERSHELL SCRIPT

  1. Download and install AWS Tools for Windows PowerShell (https://aws.amazon.com/powershell/)

  2. Create a script as shown below. In a nutshell what the script does is:

a. Execute mysqldump command (Comes with MySQL Server)

b. Zip the backup file (which reduces the size significantly)

c. Upload the zip file to S3 bucket

d. Send a notification email using SES

e. Delete the local files

This is the full script:

  • For SQL Server databases, there’s a PowerShell cmdlet called Backup-SQLDatabase but for MySQL I think the most straightforward way is using mysqldump that comes with MySQL server.

  • For password-protected zip files, you can take a look at this article (I haven’t tried it myself)

Final step: Schedule the script by using Windows Task Scheduler

This is quite straightforward. Just create a task, schedule it to how often you want to backup your database.

In the actions section, enter “powershell” as “program/script” and the path of your PowerShell script as “argument” and that’s it.

Resources