Hey everybody, I wanted to bring to your attention the changes that we made to App.HandleSpecialURL in 2014r2.1.
Please see my blog post: http://www.xojo.com/blog/en/2014/08/handlespecialurl-changes-in-2014r21.php
Thanks, Greg. Your blog post is exceptionally helpful.
Wow. So it sounds on the surface like what was working well in stand-alone and broken on CGI is uniformly broken now? This is another one of those I never would have expected by reading the release note or the linked Feedback case when beta 4 came out. Please tell me my understanding of this is completely wrong. It really makes no sense to me.
No, I believe what he’s saying is that it was always really ‘broken’. If it worked (which it did in many instances) it was purely by luck.
I believe the WebRequest was putting variables into a dictionary. WebRequests can have multiple entities with the same parameter name. Thus if you had a web request using the multiple parameters with the same name it would only keep the last value instead of all of them. So if you were lucky and only had the one parameter it ‘worked’. But it’s still technically broken because it’s not handling it properly.
It breaks code. That part sucks but I’m not sure what else they could do.
It was uniformly broken but folks relied on this brokeness
A valid url can contain multiple variables with values and in many other languages you’d get an array (Query string - Wikipedia)
ie/
http://something.com/foo?arg=1&arg=2
would result in the allied app getting an arg for arg with values 1 & 2
Cant do that with a dictionary that just returns key value pairs
Let me explain two scenarios that were completely broken:
- You receive a POST request which has both form-encoded data and querystring variables. If you happened to have the same key name in the body and in the querystring, the ones from the body would overwrite the ones from the querystring. There was no way to fix that if you didn’t control the sending code (like if you were working with a 3rd party service)
- You receive a POST that has key/value pairs in the body, but it’s in a proprietary format… perhaps not url encoded. Our mechanism was trying to guess the content-type and then if it decided it was key/value pairs it would parse it and then blow away the data.
We should have fixed this behavior when HandleSpecialURL was introduced, but we didn’t. This is the only way to satisfy everyone’s needs… including ours.
OK, wait. On stand-alone, it worked awfully close to what the LR specified. On CGI, it was a mess. Now, both are a mess. Additionally, there is no way I would have divined that from the release note and linked feedback case in beta 4. No attention drawn to this then?
I’m probably just on a completely different planet of expectations lately. Let’s see if anyone else chimes in…
Are you saying that they are not working the way I described in the blog post?
[quote=119809:@Greg O’Lone]Are you saying that they are not working the way I described in the blog post?
[/quote]
I’m sure it is. But that is nothing like what I would have expected from the release note in beta 4 or the feedback case.
That the stand-alone worked reasonably close to spec and the CGI was completely broken relative to spec was one reason I recommended stand-alone to some of my customers for web apps that served up an API. Now, I have no control of influence into whatever random things you do to “fix” the framework and that’s a risk I have to accept. But it is beyond counter-intuitive why this change was made, when it was made, and how it was rolled out. Forget that this is in a .1 release, which is widely expected to deal with regressions in a major release.
I know that one of my customers has communicated to you for about 8 months that CGI for deployment on Xojo Cloud is unworkable due to HandleSpecialURL being broken in CGI. And now you’ve broken it in stand-alone. We’re evaluating. Please make the obvious logical leap there, because your customers are.
Thanks Greg for the blog post! Glad Entity is now functioning. Was bummed in a couple projects having to use “alternative methods” to get posts, and being able to get GET requests and no POST was a bit of a challenge integrating and routing data between “alternate methods,” all while trying to keep secure data secure. Now GET and POST finally work in HandleSpecialURL. Thank you guys for all your hard work!
Great! One less reason to recommend standalone over CGI! 
This change was made because it does fix a regression in 2014r2.[quote=119814:@Brad Hutchings]I know that one of my customers has communicated to you for about 8 months that CGI for deployment on Xojo Cloud is unworkable due to HandleSpecialURL being broken in CGI. And now you’ve broken it in stand-alone. We’re evaluating. Please make the obvious logical leap there, because your customers are.[/quote]
I am not aware of anyone complaining about this issue for 8 months with regards to Xojo Cloud, but that aside it’s not broken. You can get to all of the data, all the time, and it’s consistent across deployment types. Believe me when I say that we did not make this decision lightly. We understood that it would break people’s code. What we’ve done is given you more control to process the data the way you want to, and keeping the request content separate from the QueryString the way most other languages do it.
I’m not up on web technologies so I’m probably missing something, but I don’t understand why you say broken.
Yes this change means that you have to parse it yourself, but you should know the form of the query strings your app will need to support so I don’t understand why why you say broken… You should still be able to do what you need to…
Norm,
If supporting multiple values are the issue, as dictionaries use variants, why can’t all values be an array? If there is only one value it could be an array with one element.
- Karen
It could be. The problem was that we were just stuffing values into the dictionary without considering if there was one already. Again, our internal parser worked for us, but not for many real-world scenarios.
Something is amiss here, either in the description of what changed or a detail of the explanation.
“Web: Standalone apps no longer have their Entity and QueryString munged in HandleSpecialURL.”
We never experienced this as a problem for POST queries. The Entity was correct in stand-alone, but completely inaccessible in CGI. That particular detail is what necessitated stand-alone deployment (although I’m a fan for a wide class of apps for other reasons as well). If that’s what you fixed, perhaps you documented it backwards?
The Entity is still as expected in stand-alone 2014r2.1. Perhaps that has to do with content-type? But that’s not clear from the case.
We expect JSON data in the Entity. It’s there pre-2014r2.1 and in 2014r2.1 for stand-alone.
I’ll keep my eye on this one.
Good grief. It was in beta, and we were pretty clear with specific details about why it wouldn’t work for us at the time. But whatevs.
[quote=119836:@Greg O’Lone]This change was made because it does fix a regression in 2014r2.
[/quote]
I might be mistaken, but combing through 2.1 beta release notes, I think this might be the regression you’re talking about, although it’s still marked as a beta bug?
<https://xojo.com/issue/33967>
And it doesn’t appear in the release notes for 2014r2.1. Trying to decode what has changed and why has to be really interesting for people who don’t have beta access.
P.S. Thanks for adding “release” to fixed cases in Feedback.
Because the only ones that SHOULD be arrays are those that occur more than once
The others should be accessible as simple values
IF you could test with something like
for each key in paramValuesDict.Keys
if paramValuesDict(key).IsArray
// than handle array of values for this key
else
// handle single value
end if
next
And you kind of can with varitype but its still got issues as Greg noted in the blog post with mime types etc etc
Look at this weird thing in Debug. Is it a Debug Bug, don’t?
https://drive.google.com/file/d/0B6w6JZen_vW2d3o5OUUxbTE5Znc/edit?usp=sharing
[parsing the Entity String into a Dictionary]
The misrepresentation of the Array seems a Debugger Bug.
[quote=119895:@Rick Araujo]Look at this weird thing in Debug. Is it a Debug Bug, don’t?
https://drive.google.com/file/d/0B6w6JZen_vW2d3o5OUUxbTE5Znc/edit?usp=sharing
[parsing the Entity String into a Dictionary][/quote]
Ok well you’re just trying to break this now. The string you are parsing looks like this:
a = 34 & b = a b c s%\\$s = no.bug & c=Rick A. & c=Another User & c = Another Another User & dAnd that’s not a legal string. Spaces are not allowed and you need to use EncodeURLParameter to encode the values, so it should look like this:
a=34&b=a%20b%20c%20s%\\&s=no.bug&c=Rick%20A.&c=Another%20User&c=Another%20Another%20User&dThis is not the issue. The protection against bad formation can be ignored. You can remove the spaces from the string. Check the c value in the dictionary. Debugger can’t represent arrays or the system was broken?