New Web Framework

  1. ‹ Older
  2. 8 months ago

    Brandon W

    18 Jun 2019 Pre-Release Testers, Xojo Pro Texas, USA

    If HA proxy is in front of a Xojo web app you get a ton of functionality including ssl offloading and the ability to scale your apps. There is no cost and it is fairly easy to setup and supports http2. Also it frees xojo from having to do a ton of work and in turn brings web 2.0 to us more quickly. What am i missing? Since xojo recommends this anyway, what would be the benefit of having xojo add http/2.0 support to the web framework? The web framework is for building web apps and i suspect that using it for something else is not the best idea.

  3. Derk J

    19 Jun 2019 Pre-Release Testers, Xojo Pro

    @Greg OLone That’s not really practical. Xojo Web doesn’t need the whole spec, so we’re implementing the parts that are required to call our server HTTP/1.1 and to make our framework as efficient as possible.

    There’s nothing that prevents you from creating your own HTTP server for your special needs.

    While that's true, it's harder to maintain different projects. And currently the App.HandleURL does provide "some" functionality but it's incomplete, we can't keep a connection alive, we can't flush "partial" data. It seems to be threaded but unable to "flush" so keeping alive is not possible currently. Therfore the whole http1.1 spec can't be implemented making it not very practical. Reconnecting for realtime data over http is alot of overhead.

    @Greg OLone Keep-alive aka persistent connections will just work, but we have no plans to expose server sent events or connection control as a developer might inadvertently disconnect a connection that we’re using.

    There is no need for Xojo to expose the whole http/1.1 spec. There IS a need for a Xojo user to be able to do so.

    @Greg OLone First of all, we never said that HTTP/2.0 would be part of the Web 2.0 package. To do so would require a huge refactoring over what we’ve already done and we were not prepared to delay the product any more for the unproven improvements we would get. If, as you say, there is “much more competition to this available”, and our web framework doesn’t suit your needs, my suggestion is to use what works best for you and your product.

    I didn't said http/2.0 will be in there but it PROVIDES alot of very usefull features (priority, concurrency etc.) in the spec. So if we're able to use that, then it would be a big plus as it's HTTP/1.1 backwards compatible and we won't be behind the hurdle again.

    Look around in the forums, see how many people talk about "API" (REST API's) and creating them. Having this as a complete possibility in XOJO would get more attention to XOJO itself. As it's easy to use xojo and create micro-services and such. It's scalable (especially having a stand-alone webserver) and extendible.

    Currently we are stuck with very limited possibilities or "create your own" but then using open-source alternatives is easier and provides more possibilities. We DONT want to go to other tools, we want code-reusability.

    It's not that Xojo isn't going the way we ask as App.HandleURL came by FB request and it did help alot. Just having "a little more"* to it would mean we can move along with the "tech" currently still in use.

    *allowing a developer to start a web app project and implement App.HandleURL and keep the connection open (closing when the event returns or sets a close flag. I don't think this is too much to ask?

  4. olivier v

    19 Jun 2019 Pre-Release Testers, Xojo Pro Europe (France)

    @Greg OLone First of all, we never said that HTTP/2.0 would be part of the Web 2.0 package.

    But it is an implicit expectation for a web framework that will be released around 2020, and that will probably be mature a few years later (like all frameworks).

    Even commercially, for you, it still makes a lot of sense (marketing an HTTP1.1 web framework in 2020?)... If I were you, I would prefer to delay the project for a few months and release it in http2. Because once the framework is released, it will be much more difficult to change its foundations and you will have so many other things to improve and add.

    That being said, if in reality, http2 does not make a huge difference, I understand that it is not a priority for you. But commercially, it seems to be risky. Put yourself in the shoes of new customers who will look for a new web framework and make comparisons.

    Translated with www.DeepL.com/Translator

  5. Greg O

    19 Jun 2019 Xojo Inc scout.galaxy.barn

    First of all, I didn’t say that we wouldn’t support http/2.0, just that it’s not in the current design doc. As I’ve stated in this (and last) year’s XDC session, we’re aiming to target feature sets on browsers, but that includes some older browser technologies for compatibility with older browsers for the time being... so we had to do HTTP/1.1 first.

    Something that you might not know about http/2.0 is that you must already have the http/1.1 Upgrade capability working to be able to even negotiate a 2.0 connection because it is a completely binary protocol and because so many servers assume http/1.1. So regardless, we still need the protocol we implemented to get to http/2.0 in the future anyway.

    Just to be clear, unless someone can present a solid use case that benefits a majority of our web customers, I’m not sure that we would expose the things that Derk is asking for. It was never our intention to expose the raw inner workings of the underlying web server. Mostly because doing so actually restricts our flexibility in some ways. Imagine if we’d done that already and you came to rely on an enhancement that we’d made in our 1.0 server that couldn’t be kept when we moved to 1.1 for some reason. We’d be having a totally different discussion today.

    One reason I say all this is that while RFC specs seem to be cut and dry on the surface, implementation is still done by someone’s interpretation of those specs. For instance, It’s not uncommon for http servers to behave in subtly different ways because of that and we could easily find ourselves back here in the future discussing all the things you think Xojo did wrong when implementing http/2.0 because our needs are different than yours.

    Again, not saying that we won’t do it, but the http server spec we are using is only a small piece of the overall Web Framework overhaul going on, and work on the http server for this version was completed roughly five months ago. We’ve got so many other big items still to be completed before we can call it even ready for testing, that I’m confident in saying that http/2.0 won’t be in the first new release.

    P.S. - I don’t know if you remember, but we went through a similar struggle with WebSockets a few years ago. We implemented a pre-ratified version (at users request) and found ourselves with two problems:

    1. Only two of the four major browsers implemented the version we did
    2. The overhead of translating to & from the binary protocol largely negated the speed benefits we were hoping for.
  6. Derk J

    19 Jun 2019 Pre-Release Testers, Xojo Pro

    Our use cases:

    • Xojo IOS coupling via REST/HTTP API streaming data from the server using SSE (server sent events)
    • Xojo console on linux (mainly) streaming data via REST/HTTP API
    • Xojo desktop applications communicating with (preferrably realtime HTTP streaming using the simple URLConnection class (async). These classes already allow Keep-Alive HTTP connections and SSE.

    We use them on Raspberry Pi, X64 linux, windows and MacOS all from Within XOJO ! We wanna keep the systems alive and stay mainstream (at-least-http/1.1 keep alive as requirement).

    The REST API’s are made with Xojo Web Edition (alternatively AloeExpress but Xojo web has way more clean projects and more ease of use! (Saving time and money).

    By this we can even allow more people to use Xojo because it’s easy to understand and easy to learn to people.

    @Greg OLone we really apreciate xojo as-is. As a side effect we need to stay into the race.

    We do already use services like Ably.io and PubNub for example, they provide the same but with alot more cost to it. Having Xojo Web edition to be able (DIY) to replace those with Xojo Cloud or self hosted will be a Big Plus.

  7. James S

    20 Jun 2019 Pre-Release Testers, Xojo Pro

    @Greg OLone P.S. - I don’t know if you remember, but we went through a similar struggle with WebSockets a few years ago. We implemented a pre-ratified version (at users request) and found ourselves with two problems:

    Only two of the four major browsers implemented the version we did
    The overhead of translating to & from the binary protocol largely negated the speed benefits we were hoping for.

    I remember :) The first is no longer a problem. Everyone supports it now! And I have to argue about the second. In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly. This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you can’t do both for what is best served.

    I have a huge amount of time invested in web apps. I am nervous about the change. I keep sitting down to rebuild some core things with a totally different back end, but the amount of effort is non trivial as they say ;) I’m really looking forward to getting a look at web2,0! I don’t really care what the underlying implementation is. I need faster response and more dynamic layouts without having to make my entire interface a plugin which negates most of the benefits of using a xojo web app anyway. I would also very much like some lower level access to the socket in the handleURL. Currently I cannot efficiently send large resources as they have to be totally loaded into memory before I can pass them off. I could handle my own chunked or multipart replace protocols if I could make things actually send before I had to return from the end of the handleURL handler. And no, I can’t just serve the larger movie resources from the regular apache server as there isn’t one. It’s running stand alone and everything needs to be going out on the same port or the user configuration would be untenable.

  8. Michel B

    20 Jun 2019 Pre-Release Testers, Xojo Pro
    Edited 8 months ago

    I have bumped into the WebFile limit with my download service. Past 30K or so, it was too much for Xojo.

    I now have implemented a php fpassthru(); small program which effectively serves bigger files.

    To me, Xojo Web is a bit like a small car that we are trying to boost into a rally monster.

  9. Derk J

    21 Jun 2019 Pre-Release Testers, Xojo Pro

    @James S I remember :) The first is no longer a problem. Everyone supports it now! And I have to argue about the second. In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly. This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you can’t do both for what is best served.

    I have a huge amount of time invested in web apps. I am nervous about the change. I keep sitting down to rebuild some core things with a totally different back end, but the amount of effort is non trivial as they say ;) I’m really looking forward to getting a look at web2,0! I don’t really care what the underlying implementation is. I need faster response and more dynamic layouts without having to make my entire interface a plugin which negates most of the benefits of using a xojo web app anyway. I would also very much like some lower level access to the socket in the handleURL. Currently I cannot efficiently send large resources as they have to be totally loaded into memory before I can pass them off. I could handle my own chunked or multipart replace protocols if I could make things actually send before I had to return from the end of the handleURL handler. And no, I can’t just serve the larger movie resources from the regular apache server as there isn’t one. It’s running stand alone and everything needs to be going out on the same port or the user configuration would be untenable.

    A simple fix would be to allow the connection open untill the App.HandleURL returns. It's already threaded i guess so it should not be that hard to change it (at least in my point of view). The problem now is; We can't flush, write or send anything over the "HandleURL call" (WebRequest.Print() as we have to Return before the content is actually parsed. That's not allowing us to send files, streams (like Server-Sent-Events and Chunked transfers). Thus it's closing the connection always.

    Maybe @Greg OLone thinks we all use Xojo web in combination with Javascript websites or even Xojo Web Pages? That's not the case for me. We use it as a backend to interal (could be loopback network interface and LAN) things. Also a a load-balanced realtime-streaming service providing json. When WebSockets where requested the intention was to get a WebSocket client class too. No so much in side Xojo-Web Pages or so. We don't even use that as much, what we do use alot is XOJO-TO-XOJO communications and Xojo to 3th party created software where a Xojo based Web API provider is preferred. That would help so much in fast deployment (RAD) of backend systems.

  10. Greg O

    21 Jun 2019 Xojo Inc scout.galaxy.barn
    Edited 8 months ago

    @James S In my testing the responsiveness of an app using web sockets was double that of one not using them. Those are small bits of data and move quickly.

    But they didn’t make a huge difference with the way our web apps work overall. The problem had to do with how many connections had to be open at any given time. There can be a lot of idle time in a session for which open connections jus consume resources.

    This was entirely on stand alone apps which otherwise used a separate connection for each event. And still do it seems. I suppose it was slower for sending larger resources but there is no reason you can’t do both for what is best served.

    And therein lies the problem... we don’t know when a request comes in if it’s for a large resource or a small one, because you (our customers) are writing the code which causes the responses to be created. The largest offenders were listbox (for users sending hundreds of rows of data) and new pages with lots of controls. Neither of these are going to be an issue in the new framework because listbox is always data-on-demand and controls are completely built on the browsers now.

  11. Greg O

    21 Jun 2019 Xojo Inc scout.galaxy.barn
    Edited 8 months ago

    @Derk J A simple fix would be to allow the connection open untill the App.HandleURL returns. It's already threaded i guess so it should not be that hard to change it (at least in my point of view). The problem now is; We can't flush, write or send anything over the "HandleURL call" (WebRequest.Print() as we have to Return before the content is actually parsed.

    If only it were that simple...

    When you’re writing to the response object, you’re not actually connected to the socket. That’s because we can’t guarantee that all of the headers will be set before the content is written. Imagine if a user were to write the content of a PDF file and then set the mime type. We can’t go back and insert that in the stream after the fact. Never mind the fact that http/1.0 doesn’t work that way.

    That's not allowing us to send files, streams (like Server-Sent-Events and Chunked transfers).

    Without direct stream access, you won’t be able to do server sent events through HandleURL. As far as Chunked transfers, the server code does support them (because we MUST according to the spec) but again, their use is hidden from users. That said, I’m not opposed to a feature request for some other access. It won’t be for the first release, but it’s certainly something we could look into for the future. As I said above though... having this sort of direct-write access to a socket would mean stricter rules on the order that things can be done.

    Thus it's closing the connection always.

    Currently It always closes because that’s what the http/1.0 spec says it should do. You can’t just adopt one or two things from a newer spec and expect things to work. There are several items in the http/1.1 spec which are indicated as MUST be implemented to call yourself 1.1 compliant which just weren’t possible with the original implementation. Because the spec says so, browsers have a reasonable expectation that servers have this minimum capability and things go awry when they don’t.

  12. Derk J

    21 Jun 2019 Pre-Release Testers, Xojo Pro

    Will create Feature Request for this once i have the time.

    @Greg OLone

    If only it were that simple...

    When you’re writing to the response object, you’re not actually connected to the socket. That’s because we can’t guarantee that all of the headers will be set before the content is written. Imagine if a user were to write the content of a PDF file and then set the mime type. We can’t go back and insert that in the stream after the fact. Never mind the fact that http/1.0 doesn’t work that way.

    Well that's a user's problem isn't it? If someone using app.HandleURL and wants to send a response and sends the body first then then headers, o well then that user should be reading some docs.

    I understand your point of view here, still it would be great if it's possible in the future.

  13. 2 weeks ago

    Jay M

    Feb 14 Pre-Release Testers, Xojo Pro Austin TX

    @Julian S API 2.0 Sept/Oct 2019
    Web 2.0 Christmas 2019
    Android Summer 2020

    Looks like this was about six months optimistic!

  14. Julian S

    Feb 14 Pre-Release Testers, Xojo Pro UK

    @Julian S API 2.0 Sept/Oct 2019
    Web 2.0 Christmas 2019
    Android Summer 2020

    API2 delivered Oct 2019

    Bearing in mind these were beta dates, +2-3 months for the issues with api2 renames (I didn't know the rename was going to happen when I got my crystal ball out before posting :)) would put web 2.0 to feb-march which is spot on for a xojo connect. Maybe it would have been in beta by now if xojo connect wasn't so close, who knows. We'll see if Android betas around August, cmon Travis don't make me throw out my crystal ball ;)

  15. Norman P

    Feb 14 Pre-Release Testers, Xojo Pro outside listening to the crick...

    FYI the road map moved android after iOS API 2

  16. Julian S

    Feb 14 Pre-Release Testers, Xojo Pro UK

    Oooo that's interesting, so do I predict iOS and Android releasing really close to each other, I'll get my magic 8-ball out ;)

  17. Michel B

    Feb 15 Pre-Release Testers, Xojo Pro

    We are turning off topic here.

    I suppose Android was not as complete as they wanted. Besides, iOS being closer to Mac, perhaps more code could be easily reused.

    I really wish this time they take the opportunity of API 2 in order to expose more controls properties and methods. Like controls color, background image and border, for instance. Or GetFocus and LostFocus.

    The idea being that is kind of ridiculous to attempt cross platform when half of methods and properties are missing on iOS, although the framework provides them.

  18. last week

    Seth O

    Feb 16 Pre-Release Testers, Xojo Pro Florida/England/California/NY/...
    Edited last week

    @Greg OLone As far as Chunked transfers, the server code does support them (because we MUST according to the spec) but again, their use is hidden from users. That said, I’m not opposed to a feature request for some other access.

    Something like this from 2016?

    Feedback 46315 - xojo.net.httpsocket should be able to handle chunked data transmission:  This is needed in order to communicate with various web services (Mitel's* phone API, to name just one of the multitudes).  It'd be nice to have a simple property like in C#: request.SendChunked = true;

    I fully understand hiding implementation details from users so that they can be altered or completely refactored later, but what's the point of preventing users from making use of something implemented as part of a standard? I'm confused :(

    *it was Shoretel's back in 2016

  19. Greg O

    Feb 17 Xojo Inc scout.galaxy.barn

    @Seth O Something like this from 2016?

    Feedback 46315 - xojo.net.httpsocket should be able to handle chunked data transmission:  This is needed in order to communicate with various web services (Mitel's* phone API, to name just one of the multitudes).  It'd be nice to have a simple property like in C#: request.SendChunked = true;

    First of all, you don’t use Xojo.Core.HTTPSocket when you are sending responses to an http request. They are purely client-side classes, and can handle chunked responses.

    I fully understand hiding implementation details from users so that they can be altered or completely refactored later, but what's the point of preventing users from making use of something implemented as part of a standard? I'm confused :(
    *it was Shoretel's back in 2016

    Because the web framework HTTPServer was never designed to be something that an end user would have direct access to and use separately. It is highly integrated with the Session management system and the rest the of the web framework for the purpose of speed.

  20. Walter P

    Feb 17 Pre-Release Testers, Xojo Pro
    Edited last week

    @Greg OLone Because the web framework HTTPServer was never designed to be something that an end user would have direct access to and use separately. I

    That makes sense, but I'd be hard-pressed to think of any high-level programming language that doesn't include a full-featured HTTP server as part of its standard library these days. That's actually been true for a decade or more. Which means this is a feature Xojo should have provided to its users for more than a decade now. "Roll-your-own HTTP server with sockets" is a million miles away from being a RAD tool, and (as I said) forget about RAD tools, even just plain languages—pretty much every single one other than Xojo—have provided an HTTP server out of the box for years.

or Sign Up to reply!