Amazon S3 'PUT' HTTP error

General
1

I have a need to store some data to some Amazon S3 buckets. My Xojo program successfully creates new buckets, so I’ve gotten past all the newbie encryption/formatting issues. But, when I try to store a simple text file to a bucket via a PUT transaction, I get an HTTP error, “HTTP/1.1 505 HTTP Version Not Supported”. The Xojo documentation indicates that Xojo sockets are a ‘HTTP 1.0’ type. Has anyone else come across this? There could be an obvious work-around, but I have limited experience with socket protocols.

It appears that Amazon is expecting 1.1 protocol sockets and Xojo provides version 1.0. If so, is there a Xojo-only solution? I’d rather not have to package/install another environment, such as perl, on each of the target systems in order to enable storing the files to an S3 bucket.

I’m using Xojo 2013 Release 3. Thanks in advance for your input on this. Here’s the most relevant code:

// Note:
// mySocket is a SecureSocket that was established elsewhere in the code
// AWS_Signature was created elsewhere in the code
//
dim theAuthString, hostNameString as string
dim testFileContent as string = “hello world” // force this for testing purposes
dim testFileLength as integer = len(testFileContent)

// define the Host name string
hostNameString = “myNewBucketName.s3.amazonaws.com” // force this for demo purposes

// setup headers
mySocket.ClearRequestHeaders()
mySocket.Port = 8080

// build header data
mySocket.SetRequestHeader(“Host”, hostNameString)
mySocket.requestHeaders.AppendHeader(“Date”, AWS_Date)
theAuthString = "AWS " + AWS_Key + “:” + AWS_Signature // construct authorization string
mySocket.requestHeaders.AppendHeader(“Authorization”, theAuthString)
mySocket.requestHeaders.AppendHeader(“Content-Type”, “text/plain”)
mySocket.requestHeaders.AppendHeader(“Content-Length”, str(testFileLength))
mySocket.requestHeaders.AppendHeader(“Expect”, “100-continue”)

// setup content
mySocket.SetRequestContent ( testFileContent, “text/plain”)

// send it off to Amazon Each of the variations below return the same error…
//mySocket.SendRequest(“PUT /testfile.txt HTTP/1.0”, hostNameString)
mySocket.SendRequest(“PUT /testfile.txt HTTP/1.1”, hostNameString)
//mySocket.SendRequest(“PUT /testfile.txt”, hostNameString)

2

There’s no way to make the HTTPSocket class send HTTP/1.1 as the version unless you roll your own HTTP request.

e.g.

[code]
dim theAuthString, hostNameString as string
dim testFileContent as string = “hello world” // force this for testing purposes
dim testFileLength as integer = len(testFileContent)
hostNameString = “myNewBucketName.s3.amazonaws.com” // force this for demo purposes

’ use the InternetHeaders class to construct the headers
Dim headers As New InternetHeaders
headers.SetHeader(“Host”, hostNameString)
headers.SetHeader(“Date”, AWS_Date)
theAuthString = "AWS " + AWS_Key + “:” + AWS_Signature // construct authorization string
headers.SetHeader(“Authorization”, theAuthString)
headers.SetHeader(“Content-Type”, “text/plain”)
headers.SetHeader(“Content-Length”, str(testFileLength))
headers.SetHeader(“Expect”, “100-continue”)

’ create the HTTP 1.1 request message as a plain String
’ EndOfLine.Windows is identical to the HTTP end of line marker
Dim HTTPmessage As String = “PUT /testfile HTTP/1.1” + EndOfLine.Windows

’ add the headers
HTTPmessage = HTTPmessage + headers.Source + EndOfLine.Windows + EndOfLine.Windows

’ add the message content
HTTPmessage = HTTPmessage + testFileContent

’ use the Write method to write directly to the socket
mySocket.Write(HTTPmessage)
mySocket.Flush[/code]

The HTTPSocket’s events should fire normally using this technique (tested using RS2011r4.3)

3

Great! I’ll try that. Thanks for the quick reply, Andrew!

4

I just discovered a nasty problem with this. The HTTPSocket will write a second, incomplete HTTP request to the socket immediately after the custom message is written. HTTP 1.1 allows this, but if the server supports HTTP pipelining, you may get a response to the original request followed by an error page to the second request. Setting the Connection: close header should disable pipelining on most servers, but I gather from the Expect: 100-continue header that you need to keep the socket open.

Using a bare TCPSocket will work just fine using the above code, however the TCPSocket doesn’t parse HTTP into events like PageReceived. :\

5

Xojo MUST evolve it’s HTTP component to be 1.1 compatible urgently. It’s getting unusable.
I suggest the engineers taking a look in new libraries like http://pocoproject.org/

6

I have S3 file storage working now, thanks to your helpful example Andrew. Yes, I’m seeing the socket error after each transaction. I can probably work around it with some “special case” kludges.
Yes, I agree with Rick A’s comment. Xojo is a powerful tool, but needs to keep some important functions current. I look forward to seeing HTTP 1.1 compatibility in the near future.

7

If you want HTTP 1.1 support, sign onto this feedback request: <https://xojo.com/issue/10311> . The status says “scheduled”, but that was a year ago.

8

HTTP 1.1 is just a TCPSocket that implements the protocol.
Anyone should be able to write one :stuck_out_tongue:

10

[quote=36545:@Norman Palardy]HTTP 1.1 is just a TCPSocket that implements the protocol.
Anyone should be able to write one :P[/quote]

I hope you’re not serious. If you are, then I hope you’re not speaking officially. If you’re speaking officially, then my confidence in Xojo, Inc has been severely misplaced.

11

HTTP 1.1 IS just a protocol that is implementable using a TCP socket & code (same as many other protocols).
HTTP 1.0 is as well.

Yes our support is scheduled but it depends on some other things that may not happen in a time frame that meets the needs of the OP (or anyone who needs it before we deliver it).
If someone needs this desperately prior to us having such support it IS possible to write one - grab the spec & write the code to implement the spec.
SMTP / POP3 and a LOT of other network protocols are just code to handle the protocol & a tcp socket.
Various users over the years have written many things like FTP, their own SMTP / POP, and a wide variety of other protocols - including a simple HTTP SERVER (code that talks the right protocol to Safari etc) - you use it if you RUN a web project or build standalone web apps.
I’ve written an entire DB connection package using just TCP sockets talking the right protocol to the DB server.

So “am I serious” about anyone being able to write one - yes - anyone could as long as they follow the spec.
Are many likely to - no.

While there are some things that only we can do as they require integration with the compiler or language or IDE there are LOTS of things our user base can do as its literally “just code”. This is potentially one.

12

I was referring more to the implication that “write your own” is Xojo’s official response. I have written several application protocols on top of the TCPSocket. I’m working on my own HTTP 1.1 implementation and I find it to be non-trivial; it’s more than just updating the version number. Pipelines and chunked encoding give me headaches.

I don’t think HTTP 1.1 support is an unreasonable feature request. The OP probably doesn’t want to spend time writing his own HTTP 1.1 implementation or abusing the HTTPSocket like I demonstrated: he wants to talk to a Amazon’s cloud. I’m sure a lot of Xojo’s other users would like to talk to Amazon as well. Too bad for them?

13

[quote=36545:@Norman Palardy]HTTP 1.1 is just a TCPSocket that implements the protocol.
Anyone should be able to write one :P[/quote]

This is a VERY, VERY, not acceptable response. I find it even disrespectful.

14

This is acceptable, undesirable, but acceptable.

15

[quote=36632:@Rick A.]This is a VERY, VERY, not acceptable response. I find it even disrespectful.
[/quote]

Calm down. It would make a great community open source project, no?

16

I didn’t understand your argument.

17

You’re still looking for an open source project to lead, no? This landed in your lap. Go do it!

18

It’s done. I already offered it ready to Xojo, few posts above.
It’s landed on Xojo’s lap, we don’t need more efforts transferred to the user’s responsibility.

19

[quote=36563:@Andrew L.]I was referring more to the implication that “write your own” is Xojo’s official response. I have written several application protocols on top of the TCPSocket. I’m working on my own HTTP 1.1 implementation and I find it to be non-trivial; it’s more than just updating the version number. Pipelines and chunked encoding give me headaches.
[/quote]
Please read what I wrote.
I said that IF you need it before we have one there IS the option of writing one yourself.
Nothing more.

20

You said, “anyone should be able to write one”. Since I find implementing full HTTP 1.1 support to be non-trivial, you can understand why I view your suggestion with dismay. I am by no means the “1337”-est Xojo hacker, but neither am I a complete n00b. If I have trouble implementing HTTP 1.1 for fun, how do you expect a less-experienced user on a deadline to cope at all?

21

Andrew, Please see the emoticon after Norman’s statement. Here is a more full explanation for you… When they implement an HTTP 1.1 class, it will be a subclass of TCPSocket and written entirely in Xojo. No special secret back doors or anything like that. Norman wasn’t taunting anyone.