First of all, I didnt say that we wouldnt support http/2.0, just that its not in the current design doc. As Ive stated in this (and last) years XDC session, were aiming to target feature sets on browsers, but that includes some older browser technologies for compatibility with older browsers for the time being… so we had to do HTTP/1.1 first.
Something that you might not know about http/2.0 is that you must already have the http/1.1 Upgrade capability working to be able to even negotiate a 2.0 connection because it is a completely binary protocol and because so many servers assume http/1.1. So regardless, we still need the protocol we implemented to get to http/2.0 in the future anyway.
Just to be clear, unless someone can present a solid use case that benefits a majority of our web customers, Im not sure that we would expose the things that Derk is asking for. It was never our intention to expose the raw inner workings of the underlying web server. Mostly because doing so actually restricts our flexibility in some ways. Imagine if wed done that already and you came to rely on an enhancement that wed made in our 1.0 server that couldnt be kept when we moved to 1.1 for some reason. Wed be having a totally different discussion today.
One reason I say all this is that while RFC specs seem to be cut and dry on the surface, implementation is still done by someones interpretation of those specs. For instance, Its not uncommon for http servers to behave in subtly different ways because of that and we could easily find ourselves back here in the future discussing all the things you think Xojo did wrong when implementing http/2.0 because our needs are different than yours.
Again, not saying that we wont do it, but the http server spec we are using is only a small piece of the overall Web Framework overhaul going on, and work on the http server for this version was completed roughly five months ago. Weve got so many other big items still to be completed before we can call it even ready for testing, that Im confident in saying that http/2.0 wont be in the first new release.
P.S. - I dont know if you remember, but we went through a similar struggle with WebSockets a few years ago. We implemented a pre-ratified version (at users request) and found ourselves with two problems:
0. Only two of the four major browsers implemented the version we did
0. The overhead of translating to & from the binary protocol largely negated the speed benefits we were hoping for.
Xojo IOS coupling via REST/HTTP API streaming data from the server using SSE (server sent events)
Xojo console on linux (mainly) streaming data via REST/HTTP API
Xojo desktop applications communicating with (preferrably realtime HTTP streaming using the simple URLConnection class (async). These classes already allow Keep-Alive HTTP connections and SSE.
We use them on Raspberry Pi, X64 linux, windows and MacOS all from Within XOJO ! We wanna keep the systems alive and stay mainstream (at-least-http/1.1 keep alive as requirement).
The REST APIs are made with Xojo Web Edition (alternatively AloeExpress but Xojo web has way more clean projects and more ease of use! (Saving time and money).
By this we can even allow more people to use Xojo because its easy to understand and easy to learn to people.
@Greg O’Lone we really apreciate xojo as-is. As a side effect we need to stay into the race.
We do already use services like Ably.io and PubNub for example, they provide the same but with alot more cost to it. Having Xojo Web edition to be able (DIY) to replace those with Xojo Cloud or self hosted will be a Big Plus.
[quote=442031:@Greg O’Lone]P.S. - I dont know if you remember, but we went through a similar struggle with WebSockets a few years ago. We implemented a pre-ratified version (at users request) and found ourselves with two problems:
Only two of the four major browsers implemented the version we did
The overhead of translating to & from the binary protocol largely negated the speed benefits we were hoping for.[/quote]
I remember The first is no longer a problem. Everyone supports it now! And I have to argue about the second. In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly. This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you cant do both for what is best served.
I have a huge amount of time invested in web apps. I am nervous about the change. I keep sitting down to rebuild some core things with a totally different back end, but the amount of effort is non trivial as they say Im really looking forward to getting a look at web2,0! I dont really care what the underlying implementation is. I need faster response and more dynamic layouts without having to make my entire interface a plugin which negates most of the benefits of using a xojo web app anyway. I would also very much like some lower level access to the socket in the handleURL. Currently I cannot efficiently send large resources as they have to be totally loaded into memory before I can pass them off. I could handle my own chunked or multipart replace protocols if I could make things actually send before I had to return from the end of the handleURL handler. And no, I cant just serve the larger movie resources from the regular apache server as there isnt one. Its running stand alone and everything needs to be going out on the same port or the user configuration would be untenable.
[quote=442321:@James Sentman]I remember The first is no longer a problem. Everyone supports it now! And I have to argue about the second. In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly. This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you cant do both for what is best served.
I have a huge amount of time invested in web apps. I am nervous about the change. I keep sitting down to rebuild some core things with a totally different back end, but the amount of effort is non trivial as they say Im really looking forward to getting a look at web2,0! I dont really care what the underlying implementation is. I need faster response and more dynamic layouts without having to make my entire interface a plugin which negates most of the benefits of using a xojo web app anyway. I would also very much like some lower level access to the socket in the handleURL. Currently I cannot efficiently send large resources as they have to be totally loaded into memory before I can pass them off. I could handle my own chunked or multipart replace protocols if I could make things actually send before I had to return from the end of the handleURL handler. And no, I cant just serve the larger movie resources from the regular apache server as there isnt one. Its running stand alone and everything needs to be going out on the same port or the user configuration would be untenable.[/quote]
A simple fix would be to allow the connection open untill the App.HandleURL returns. It’s already threaded i guess so it should not be that hard to change it (at least in my point of view). The problem now is; We can’t flush, write or send anything over the “HandleURL call” (WebRequest.Print() as we have to Return before the content is actually parsed. That’s not allowing us to send files, streams (like Server-Sent-Events and Chunked transfers). Thus it’s closing the connection always.
Maybe @Greg O’Lone thinks we all use Xojo web in combination with Javascript websites or even Xojo Web Pages? That’s not the case for me. We use it as a backend to interal (could be loopback network interface and LAN) things. Also a a load-balanced realtime-streaming service providing json. When WebSockets where requested the intention was to get a WebSocket client class too. No so much in side Xojo-Web Pages or so. We don’t even use that as much, what we do use alot is XOJO-TO-XOJO communications and Xojo to 3th party created software where a Xojo based Web API provider is preferred. That would help so much in fast deployment (RAD) of backend systems.
But they didnt make a huge difference with the way our web apps work overall. The problem had to do with how many connections had to be open at any given time. There can be a lot of idle time in a session for which open connections jus consume resources.
And therein lies the problem… we dont know when a request comes in if its for a large resource or a small one, because you (our customers) are writing the code which causes the responses to be created. The largest offenders were listbox (for users sending hundreds of rows of data) and new pages with lots of controls. Neither of these are going to be an issue in the new framework because listbox is always data-on-demand and controls are completely built on the browsers now.
When youre writing to the response object, youre not actually connected to the socket. Thats because we cant guarantee that all of the headers will be set before the content is written. Imagine if a user were to write the content of a PDF file and then set the mime type. We cant go back and insert that in the stream after the fact. Never mind the fact that http/1.0 doesnt work that way.
Without direct stream access, you wont be able to do server sent events through HandleURL. As far as Chunked transfers, the server code does support them (because we MUST according to the spec) but again, their use is hidden from users. That said, Im not opposed to a feature request for some other access. It wont be for the first release, but its certainly something we could look into for the future. As I said above though… having this sort of direct-write access to a socket would mean stricter rules on the order that things can be done.
Currently It always closes because thats what the http/1.0 spec says it should do. You cant just adopt one or two things from a newer spec and expect things to work. There are several items in the http/1.1 spec which are indicated as MUST be implemented to call yourself 1.1 compliant which just werent possible with the original implementation. Because the spec says so, browsers have a reasonable expectation that servers have this minimum capability and things go awry when they dont.
Will create Feature Request for this once i have the time.
@Greg O’Lone
Well that’s a user’s problem isn’t it? If someone using app.HandleURL and wants to send a response and sends the body first then then headers, o well then that user should be reading some docs.
I understand your point of view here, still it would be great if it’s possible in the future.
Bearing in mind these were beta dates, +2-3 months for the issues with api2 renames (I didn’t know the rename was going to happen when I got my crystal ball out before posting :)) would put web 2.0 to feb-march which is spot on for a xojo connect. Maybe it would have been in beta by now if xojo connect wasn’t so close, who knows. We’ll see if Android betas around August, cmon Travis don’t make me throw out my crystal ball
I suppose Android was not as complete as they wanted. Besides, iOS being closer to Mac, perhaps more code could be easily reused.
I really wish this time they take the opportunity of API 2 in order to expose more controls properties and methods. Like controls color, background image and border, for instance. Or GetFocus and LostFocus.
The idea being that is kind of ridiculous to attempt cross platform when half of methods and properties are missing on iOS, although the framework provides them.
Feedback 46315 - xojo.net.httpsocket should be able to handle chunked data transmission: This is needed in order to communicate with various web services (Mitel's* phone API, to name just one of the multitudes). It'd be nice to have a simple property like in C#: request.SendChunked = true;
I fully understand hiding implementation details from users so that they can be altered or completely refactored later, but what’s the point of preventing users from making use of something implemented as part of a standard? I’m confused
[quote=475672:@Seth Ober]Something like this from 2016?
Feedback 46315 - xojo.net.httpsocket should be able to handle chunked data transmission: This is needed in order to communicate with various web services (Mitel's* phone API, to name just one of the multitudes). It'd be nice to have a simple property like in C#: request.SendChunked = true;
[/quote]
First of all, you dont use Xojo.Core.HTTPSocket when you are sending responses to an http request. They are purely client-side classes, and can handle chunked responses.
[quote]
I fully understand hiding implementation details from users so that they can be altered or completely refactored later, but what’s the point of preventing users from making use of something implemented as part of a standard? I’m confused *it was Shoretel’s back in 2016[/quote]
Because the web framework HTTPServer was never designed to be something that an end user would have direct access to and use separately. It is highly integrated with the Session management system and the rest the of the web framework for the purpose of speed.
That makes sense, but I’d be hard-pressed to think of any high-level programming language that doesn’t include a full-featured HTTP server as part of its standard library these days. That’s actually been true for a decade or more. Which means this is a feature Xojo should have provided to its users for more than a decade now. “Roll-your-own HTTP server with sockets” is a million miles away from being a RAD tool, and (as I said) forget about RAD tools, even just plain languagespretty much every single one other than Xojohave provided an HTTP server out of the box for years.
The first is no longer a problem. Everyone supports it now! And I have to argue about the second. In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly. This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you cant do both for what is best served.
Im really looking forward to getting a look at web2,0! I dont really care what the underlying implementation is. I need faster response and more dynamic layouts without having to make my entire interface a plugin which negates most of the benefits of using a xojo web app anyway. I would also very much like some lower level access to the socket in the handleURL. Currently I cannot efficiently send large resources as they have to be totally loaded into memory before I can pass them off. I could handle my own chunked or multipart replace protocols if I could make things actually send before I had to return from the end of the handleURL handler. And no, I cant just serve the larger movie resources from the regular apache server as there isnt one. Its running stand alone and everything needs to be going out on the same port or the user configuration would be untenable.
