Avoiding the 401 authorization required error with Google calendar API

Using the previously-described procedure to interact with Google Calendar's API from Visual Basic 6 led to a seemingly strange phenomena at first.

Whilst trying to add an event for the first time, during the second POST operation (which includes sending the Auth code to http://www.google.com/calendar/feeds/default/private/full) rather than the event being added, Google replied with a 401 Authorization Required error like this:

HTTP/1.1 401 Authorization required
WWW-Authenticate: GoogleLogin realm="https://www.google.com/accounts"
Cache-control: private
Content-Length: 167
Date: <whatever>
Content-Type: text/html
Server: GFE/1.3

despite the fact a valid auth code was sent. The event was therefore not added.

However, any subsequent attempts to add an event with the exact same procedure did succeed. You could then add as many events as you want. However, if you exited the program and started it up again, then, again, the first time you try and add an event it fails with a 401. Subsequent times were fine. Craziness?

Actually there is an explanation. Many thanks to Frank from the Google Calendar Data API Google group explaining the following and providing the idea to get round it.

As you can probably guess, Google, what with their 99 squillion users, own more than one server. When you log in to Google.com, you could be on any one of several physical servers. Google does this by sending a 302 (temporary redirection) header to your web browser to tell it where to look. This happens - generally - without you having the chance to notice.

What is happening here though, is Google is sending your VB6 program the 302. The library underlying the Microsoft Internet Transfer Control (wininet.dll) handles this for you and requests the URL that Google tells you to use. However, it does not forward the Authorization header with your auth code in. Result? The "proper" Google server gets your post request, but it doesn't get your auth code. Hence the 401 error.

The second time you try the same procedure, your computer already knows what specific Google server to use and communicates with it directly, not needing to go through the 302 procedures of Google telling it where to go. The headers are sent as they should be, and everything works.

OK, that explains it - but given the Internet Transfer Control wininet shields you from the redirection, what can you do about it other than make each request twice just in case?

Helpful Frank provided the following snippet of undocumented info. If you add the 'X-If-No-Redirect' header set to true during the interaction where you first communicate the auth token, it will cause a 412 (Precondition Failed) error as opposed to the 401 like this:

HTTP/1.1 412 Precondition Failed
Set-Cookie: S=calendar=abcdefghijk
X-Redirect-Location: http://www.google.com/calendar/feeds/default/private/full?gsessionid=abcdefghijk
Cache-control: private
Content-Length: 161
Date: <whatever>
Content-Type: text/html
Server: GFE/1.3

The Google server's response headers will include a X-Redirect-Location: header in with a URL including a Google session ID. The example above shows the id as "abcdefghijk". If you use this URL to add your event instead of the plain http://www.google.com/calendar/feeds/default/private/full one, you won't lose your authorisation at any point.

In VB, using the Internet Transfer Control, you can do this as below. Build up and send the POST request with the X-If-No-Redirect header like this:

strURL = "http://www.google.com/calendar/feeds/default/private/full"
strFormData = EventCode
strHeaders = "Authorization: GoogleLogin auth=" & AuthCode & _
    "Content-Type:application/atom+xml" & vbCrLf & "X-If-No-Redirect:True"   
Inet1.Execute strURL, "POST", strFormData, strHeaders

Capture the headers from the Google server's response. The last-received server header is available to your program via the Inet1.GetHeader property.

You then need to extract the URL from the X-Redirect-Location header. You could do that like this (splitting the lines from the server response into an array and searching for the correct one):

returnedHeadersArray = Split(Inet1.GetHeader, vbCrLf)   'split the server's response up into an array of string lines
For counter = 0 To UBound(returnedHeadersArray)
    If (Left(returnedHeadersArray(counter), 20) = "X-Redirect-Location:") Then  'search for the X-Redirect-Location header
        strURL = Right(returnedHeadersArray(counter), Len(returnedHeadersArray(counter)) - 21)  'store its content
        Exit For
    End If
Next counter

Then resend the original request (without the X-If-No-Redirect header) using the new URL as below:

strHeaders = "Authorization: GoogleLogin auth=" & AuthCode & _
    "Content-Type:application/atom+xml"
Inet1.Execute strURL, "POST", strFormData, strHeaders

Job done. Now your program should post an event every single time you ask it to, instead of every time except the first!

Attached is an updated copy of the example program and source code to reflect the above changes.


AttachmentSize
Google Add Event program source code v25.6 KB
Google Add Event program v240 KB

Comments

uable to post an event

Firstly a zillion thanks for the neatly described article!.The Google API page was horribly documented (http://code.google.com/apis/accounts/AuthForInstalledApps.html) specially regarding that redirect thing.

I looked up your scource code and it was quite easy to understand but when i run the v2 exe i get the following eror
http://img468.imageshack.us/img468/1117/errorvb8.jpg even after repeated attempts.

I have yet another doubt.The code you just mentioned above is authenticating me perfectly and i'm getting the headers containing the Authcode,LID and SLID.Now suppose i want to open my mail in my browser What URL must i specify coz its mentioned in the api page that i can use the Authcode to access user specific content.(I changed the orriginal code's service to "mail" from "cl" for the second scenario).

Keeping my fringers crossed for a positive reply
cheers
l'lwiki

Posting an event

Hi,

Glad you found the article a bit helpful. I agree the Google docs aren't the greatest, I have to admit a lot of my vague attempts were trial and error.

About the error you get (400 Bad Request), I have just tried to replicate it, and the only time it appears to happen is if the time isn't in the right format. It has to be in the format hh:mm:ss - and the seconds are important. So "18:41:55" should be fine, but if you entered "18:41" that would give you the error you describe. It is a bit annoying for a user to have to type that in usually so presumably if you were writing a more advanced program, you'd do something like get your program to check the format and add the seconds on before trasmitting to Google in that case.

Also check the date is right. It should be dd/mm/yyyy (i.e. 24/09/2006) (I am doing it from the UK).

Lastly I don't think the Google Apps For Your Domain calendar is fully set up to let you access the API yet, so just do it on a "normal" Google calendar if you weren't already.

About Gmail - I don't think you can do it this way as to the best of my knowledge Google hasn't released a Gdata type API for Gmail yet. Correct me if I'm wrong! I believe so far they only work for Blogger, Base and Calendar.

However, John Vey has apparently created his own open-source Gmail API package if you are OK using the .Net framework (I haven't tried yet), and there are other APIs people have written for other languages including Java, Python and Perl I think. I've no experience of any of them but if they are suitable for your needs it might be quicker than waiting for Google to produce an "official" one!

Good luck.

Adam

Event posting

Ohh thanks for replying adam,I tried the gmailer agent by johnvey but in the revision history he says ::
"Update 2/27/2005: The API is currently broken, due to Gmail's protocol change. I don't have time to work on the project, but I have received some fixes that I will review as soon as possible. -JH"

My actual aim was to retreive the authcode and then proceed to load the desired page(in my case its orkut but i asked you about gmail thinking you might not have heard of it).

So now this is what i decided i can do.First take the user info then verify if thats correct .If so then i can directly navigate to the login page and login with the user name and pass but again i will be using a webbrowser control to load the URL which means slow loading time and consumption of resources...So can u guide me on how to log onto a site with redirects and finally retreive the content using either a inet control or winsock control

Cheers
l'lwiki

Orkut API

Hi,

Sorry, didn't realise the gmailer agent didn't work. I am indeed not so familiar with Orkut, I know of it, what it is, but never used it. Apparently it also has no API at present (which seems to annoy some bloggers according to a quick google search!). Apparently automating anything on it could be against the terms of service, but then again I suppose it is for gmail and that doesn't stop anyone :-)

So I think your only option would indeed be what you say, basically go through the normal login process one way or another as though you were doing it manually via a browser. I imagine the inet control could help you do this as it can send any headers and form content (e.g. strHeaders and strFormData in my code from memory) through it and use VB or whatever to parse the result. The question is basically what do you need to send!

I can't see any quick docs on this anywhere, but I bet (like gmail) it involves cookies. I can't see much info on the web at first sight, so my best idea for what you could do is use a http sniffer or some other such program that allows you to see your web traffic. If you have one of these running then you go and manually log onto Orkut hopefully it would catch all the behind-the-scenes traffic as to what actually happens with cookies, redirects and so on. Then if it was vaguely human-readable you could reproduce it in VB? Just a thought. You should be able to "fake" cookies by inserting the "Cookie" header in the headers you send via whatever, in the format described here.

Adam

Hey adam

dear Adam
Sorry was really busy with my orkut project.I tried the cookie thing but didnt work that way .Anyway iam continuing with my webbrowser strategy right now.

But i got some thing interesting for you,a gmail notifier.Here's what i did
-=============Begin of code==============----------------------
Dim WinHttpReq As WinHttp.WinHttpRequest
Const HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0
Const HTTPREQUEST_SETCREDENTIALS_FOR_PROXY = 1

Private Sub Command3_Click()
WinHttpReq.open "GET", "https://mail.google.com/mail/feed/atom", False
WinHttpReq.send

'MsgBox WinHttpReq.Status
' Set the user name and password.

WinHttpReq.SetCredentials "google username", "google password", HTTPREQUEST_SETCREDENTIALS_FOR_SERVER
WinHttpReq.send
Text2.Text = WinHttpReq.Status & " " & WinHttpReq.statusText
Text1.Text = WinHttpReq.responseText
Text3.Text = WinHttpReq.getAllResponseHeaders
End Sub

Private Sub Command4_Click()
Dim oDom As New MSXML2.DOMDocument40
Dim oTables As IXMLDOMNodeList
Dim oChild As IXMLDOMNode
Dim bOK As Boolean
Dim sxml As String
sxml = Text1.Text
'// Load xml into Dom object
bOK = oDom.loadXML(sxml)

Call LoadTreeFromXML(oDom, tvtreeview, True)

End Sub

Private Sub Form_Load()
' Create an instance of the WinHttpRequest object.
Set WinHttpReq = New WinHttpRequest
End Sub

-==========Module Code===========----------
Option Explicit

Private Const COMMENT_COLOR = &H8000&
Private Const ATTRIBUTE_COLOR = &H808080
Private Const ELEMENT_COLOR = &H0
Private Const VALUE_COLOR = &HFF0000
Private Const PROCESSING_INSTRUCTION_COLOR = &H80FF

Public Sub LoadTreeFromXML(objDOM As MSXML2.DOMDocument, _
tvtreeview As TreeView, blnAttribNodes As Boolean)

Dim nod As MSXML2.IXMLDOMNode
Dim nodx As Node

tvtreeview.Nodes.Clear

Set nodx = tvtreeview.Nodes.Add(, , , "XML DOM v2")
nodx.Expanded = True
For Each nod In objDOM.childNodes
DisplayNode2 nod, nodx, tvtreeview, blnAttribNodes
Next

Set nod = Nothing
Set nodx = Nothing
End Sub

Public Sub DisplayNode2(objNode As MSXML2.IXMLDOMNode, objtvParentNode As Node, _
tvtreeview As TreeView, blnAttribNodes As Boolean)

Dim nod As IXMLDOMNode
Dim atrib As IXMLDOMAttribute
Dim nodx As Node
Dim noda As Node

Select Case objNode.nodeType

Case NODE_ELEMENT
Set nodx = tvtreeview.Nodes.Add(objtvParentNode, tvwChild, , _
objNode.baseName)
nodx.ForeColor = ELEMENT_COLOR
nodx.Expanded = True

Case NODE_COMMENT
Set nodx = tvtreeview.Nodes.Add(objtvParentNode, tvwChild, , _
"* " & objNode.Text & " *")
nodx.ForeColor = COMMENT_COLOR
nodx.Expanded = True

Case NODE_TEXT
Set nodx = tvtreeview.Nodes.Add(objtvParentNode, tvwChild, , _
objNode.Text)
nodx.ForeColor = VALUE_COLOR
nodx.Expanded = True

Case Else
Set nodx = objtvParentNode
End Select

If Not objNode.Attributes Is Nothing Then
For Each atrib In objNode.Attributes
Select Case atrib.nodeType
Case NODE_ATTRIBUTE
If blnAttribNodes Then
Set noda = tvtreeview.Nodes.Add(nodx, tvwChild, , _
"[" & atrib.baseName & "=" & atrib.Value & "]")
noda.ForeColor = ATTRIBUTE_COLOR
Else
nodx.Text = nodx.Text & _
" [" & atrib.baseName & "=" & atrib.Value & "]"
End If

Case NODE_PROCESSING_INSTRUCTION
Set noda = tvtreeview.Nodes.Add(nodx, tvwChild, , _
"{{" & atrib.baseName & "=" & atrib.Value & "}}")
noda.ForeColor = PROCESSING_INSTRUCTION_COLOR
End Select
Next
End If

If objNode.childNodes.length > 0 Then
For Each nod In objNode.childNodes
DisplayNode2 nod, nodx, tvtreeview, blnAttribNodes
Next
End If
End Sub

-==============End of Code=========-------------

Add a MSXML3 and ms winhttp services 5.1 in your refernces.Also add a treeview control(tvtreeview) ,three textboxes
and 2 command buttons.

I think this is the principle on which the official gmail notifier works too...

p.s:Ohh yes i'm working on a small tut on how to customize the webbrowser control to the max.i will be posting it when its ready..

See you then
cheers
lw

Very interesting!

Hi,

Sorry the Orkut cookie didn't work out for you, but thanks for posting your gmail notifier code. Very interesting! I have never actually used the winhttp object before but I see it is probably more comprehensive and wider-available than the Internet Transfer Control?

(For anyone else reading, there is some winhttp documentation at Microsoft's developer site.)

Unfortunately at this moment I don't have VB installed to compile an exectuable version. Do you have a version available for download anywhere just so we can see what it looks like? If you don't but you would like to I could certain give you an account here that allows attachments if you're interested.

Looking forward to seeing your forthcoming tutorial!

Best wishes,

Adam

Re:Hey adam

Thats really nice of you to offer me an account with attachement facilities.
I dont know how far i can make it but i'm too busy these days(or you could say i'm too lazy ;)) but i will definetely be in touch with you and regarding the tut ,that should be ready in a week or so.

btw heres the link to the self executable archive containing the project files and the exe.It uses windows common controls so i dont think it will run without the proper version installed.Anyway here's the link.

http://www.4shared.com/file/4242544/a82c1782/gmail-alert.html

p.s:regarding the account thing...You could send in the details to my email that i have mentioned.

Cheers
lw

Working well!

Hi,

Thanks for the link to the archive, I tried it out and it works great just with the default XP controls I have on my computer. I guess MSXML must come built in these days, it's been a while since I used it myself and I'm sure end-users used to have to download it seperately then.

I'll email you with the details in case you are interested. Just ignore if you don't have a use for them.

Adam

Many thanks for this

Many thanks for this article. I spent hours trying to retrieve a calendar feed from Google before I found this page which explains everything. THANKS again!

Laurent

hcgouerikv

Hello! Good Site! Thanks you! lqrzujageszsx

igxpqsxjqj

Hello! Good Site! Thanks you!

solved 400 error message

Many thanks to Ken who wrote in with the following information regarding what to do if you get "400" errors when using this type of code. Reproduced with his permission:

Like others I was getting a 400 error when running your sample exe program and again in my VB6 project which was based on your posted code.

I think I found the cause of the error and wanted to share what I found. I hope its a real solution and that I just didn't have one lucky submission. Basically I changed the time section from:

startTime='" & formattedDate & "T" & txtTimeFrom.Text & tzoffset
to:
startTime='" & formattedDate & "T" & Format(txtTimeFrom.Text, "hh:mm:ss")
& ".000Z"

and:
formattedDate & "T" & txtTimeTo.Text & tzoffset

to:

formattedDate & "T" & Format(txtTimeTo.Text, "HH:MM:SS") & ".000Z"

Time was coming through with SS as described on your site but with a space and a PM/AM designation plus the hour format did not have a preceding zero if the time was, say 1:15:23 PM.

Hope this helps someone else.

lwrqestmis

Wow, cool man, big thanks!

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote> <del> <p>
  • Lines and paragraphs break automatically.
  • You may post code using <code>...</code> (generic) or <?php ... ?> (highlighted PHP) tags.
  • You may use [acidfree:xx] tags to display acidfree videos or images inline.
  • Images can be added to this post.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
1 + 4 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.