Hello,
This is post of the onging discussion from “Real Software” forum at link below, having the same subject/title (Stack overflow on Linux, not on Windows for same source code).
http://forums.realsoftware.com/viewtopic.php?f=8&t=47890&sid=081d198a28339337f0de90abe51cd9a8
Thanks,
Shahid.
You got two really good answers there.
Hello,
Thank you. Those two points have already been discussed in the previous posts and I have asked the question that what qualifies as “modifying UI” and what is the workaround when code is changed to use threads.
With support of Windows, Linux and MAC OS in RB, and now in Xojo, there should be no requirement imposed on the developer to accommodate the smaller stack size available to same source code in Linux by using threads when that same source code without threads works fine on Windows while processing the same input data which causes StackOverflow exception on Linux.
In one of the previous posts it has been shown that the stack size available to RB applications (and now also confirmed on Xojo) on Linux is about one fourth when compared to Windows by using a simple recursive application that is allowed to cause a StackOverflow exception. This is too much of a difference in stack size that Linux OS/RB/Xojo provides to the RB/Xojo applications when compared to Windows.
It is hoped that a member of the RS/Xojo team may post technical details here about the various stack sizes available and used by RS/Xojo applications and who controls and can change those stack sizes (OS, RS/Xojo, developer, application user and so on) and any technical reference documents that already discuss such discrepancy in the stack sizes when compared between Linux and Windows and may be MAC OS as well.
Thanks and Regards,
Shahid.
A few points…
Recursive algorithms might be fine for screwing around, but usually should be avoided for production code. So asking for all these details you ask for to support a “not the best” practice, might be asking too much.
A recursive algorithm that messes with the UI, and thus doesn’t stand up to the new thread/UI rules, could be done a lot better. If you’re interested in doing it better, I’m sure there are a few people who would gladly point you in the right direction.
Third, have you Googled about how Linux assigns the app stack? A quick search pointed me to “ulimit -s”:
http://ss64.com/bash/ulimit.html
Try opening shell, setting a large stack, then launching your app from the shell. If that works for you, maybe a launch script would be a better way to start your app than double-clicking.
Hello,
The system limits of RB/Xojo must have been documented somewhere and the various stack sizes afforded to the RB/Xojo applications, I guess, should also be on that list. This is not much being asked here. Imagine a developer’s plight, having to handle the “Integer” or “String” data types on Windows, Linux and MAC OS at low level separately on each OS because the high level language being used does not provide such basic data types. Having the same (or at least comparable) stack size on all the supported OSs also seems a basic requirement that should be provided for by RB/Xojo to its application across all such supported OSs.
The size being dealt with here is more 10MBytes of VB6 source code that was migrated to RB for the sole purpose of having Linux support. Only to find out that the migrated RB code works fine on Windows, however it causes StackOverflow on Linux when decoding the same input data. With this source code, the option of using threads is not so easy to implement and neither is the option to change from recursion to iteration.
This is the only large application that I have written and I am not a professional programmer and still remain a novice, even though this application has an ASN.1 BER, BASIC-PER-ALIGNED and BASIC-PER-UNALIGNED Interpreter that has been written from scratch in VB6 and now migrated to RB/Xojo. So I have not been aware of “not to use the recursive code”.
Keeping in view the more than 10MBytes size of the VB6 code that has been migrated to RB, one of the members of the RS/Xojo team made a kind offer to obtain a copy of that code under NDA and see what solution could be found for the reported problem, however that offer is no longer outstanding as can be seen in one the posts on the “Real Software” forum on this subject.
The use of “setrlimit” with “Soft Declare” was suggested on the “Real Software” forum on this subject and was tried (RB code posted in “Real Software” forum) and it seemed to change the stack size, however the application still crashed at the same depth of recursion on Linux. I do not know much Linux and the “ulimit” was not found on SuSE SLES 11 OS so “ulimit” could not be used from command line.
Thanks and Regards,
Shahid.
My quick search that found ulimit tells me that limits such as maximum open files and default application stack size are system settings, not application settings. So I’m not sure what they could tell you.
Hello,
The stack size that “ulimit” can change from command line is most likley for the same stack that “setrlimit” can set. In the “Real Software” forum “William Yu” had suggested to “programmatically set the stack size by using a declare to setrlimit.” and that was successfully tried and both the “getrlimit” and “setrlimit” seem to work fine and the stack size was increased from 8MBytes to 32MBytes and even 64MBytes however the Linux version of the application still caused the StackOverflow exception at same call stack depth.
This seems to shows that there is most likely “some other” stack size involved here on Linux. The question now is that how that stack size can be increased without using the threads.
Regards,
Shahid.
It can’t. Not without a change to the Xojo framework. Submit a Feature Request through the Feedback app. Until then, you have 2 workarounds to choose from,
Hello,
Thank you. That is very interesting if that indeed is the case, the need to change the Xojo framework.
It should not require any Feature Request because RS/Xojo claim to support Windows, Linux and MAC OS and stack is one of the basic requirements for any application. The RS/Xojo technical support and two members of their technical team already know about this.
The current 1:4 ratio of stack sizes between Linux and Windows shows too much os a discrepancy and the developer should not be expected to accommodate this oversight of Xojo system architects through the breaking and then debugging on Linux of otherwise working source code.
Thanks and Regards,
Shahid.
But there’s a balance. If the app pre-allocates way too much stack space, that ties up resources. Recursion is nice for messing around, not a good design technique for apps that are expected to do anything useful. In a sense, you’d be asking for them to inconvenience developers who use the stack within reason so that your app will run without modification. I’d be pretty against this, if asked.
Longer term, they are switching compiler technologies to LLVM. That will likely affect stack usage in a bigger way than tuning they might do to address your concern. I guess what I’m saying is that if you want your app to run on Linux, your best bet is to do the work to develop it better with non-recursive techniques.
It’s worth noting that unlike Mach-O and PE32, stack size is not specified in ELF binaries. On Linux, the main thread’s stack size is determined by the environment you’re running in. You can see what the current setting is by running “ulimit -a” in your terminal.
Since the main stack is allocated on program launch (before you hit your code), touching it from within the process doesn’t give that process more stack space. It would, I think, let any subprocesses you spawn have more stack space. You might try having a shell script that invokes ulimit to raise the stack size and then launches your application.
Hello,
Thank you “Joe Ranieri”.
Logged in a “root” on SuSE SLES 11 OS on HP Proliant x86 server, the stack size was successfully set to unlimited by using command “ulimit -Ss unlimited” and the Linux version of the application still crashed after less than 2000 recursions when launched from the same command prompt where the stack size was increased. So it seems that there may be some other factors at play here.
Please note that the same application causes StackOverflow on Windows after more than 8000 recursions. This is with reference to the sample code posted “Sat May 18, 2013 3:53 pm” at http://forums.realsoftware.com/viewtopic.php?f=8&t=47890&sid=081d198a28339337f0de90abe51cd9a8 ?
Please advise on how to proceed further.
Thanks and Regards,
Shahid.
It should be apparent by now that Xojo is going to have to do something on their end to address this. Whether you consider it a bug report or a feature request is up to you, but Feedback is the correct channel to use to report it. The fact that several Xojo engineers have commented on it and are aware of it does not mean it will ever get into their workload. File a Feedback report. That said, the earliest you could expect a fix would be 3 months, and it really is doubtful that it would get scheduled that soon, since it doesn’t affect a lot of people.
You really should avail yourself of one of the workarounds (and I do agree with Brad about using a non-recursive solution) and get on with your project.
You have a few options - some you have already dismissed
But those are bout the only options
Saying “yes but Xojo should …” is all well & good but IF you want a shippable version of your tool on Linux you will need to do “something”
Pick an option and start working on it
To be honest, it may not be something that can be addressed on our end. It’s certainly something we can investigate in more depth to see.
I take this back, it does look like a bug on our part in calculating the available stack space on the main thread. I don’t have any immediate workaround for you, sorry.
The exact size will still remain out of our hands, but it should be made to respect the environment.
Hello,
Thank you “Tim Hare”, “Norman Palardy” and “Richard “Brad” Hutchings” for pointing out the possible solutions in this case. Due to the lack of time and effort to break and debug the working code in Linux, threads or iterations have not been tried, particularly keeping in view that the same code works fine on RB/Xojo in Windows.
Thank you “Joe Ranieri” for indicating that this may be a bug. I do not have a license so most likely I will not be able to send a feedback or a bug report. So how will this be handled by RS/Xojo please? If this is fixed in due time then would it be possible to allow the “setrlimit” call to be taken in to account (that “William Yu” had suggested) to increase the Linux stack size? So that there is no need to run ulimit command from a script to launch this application and the required stack size is programmatically set to the value needed from within the application itself.
Thanks and Regards,
Shahid.
Saying “the same code works fine on RB/Xojo in Windows” is a little misleading. You just haven’t encountered a sufficiently large or complex example. It suffers from the same limitation, you just don’t hit it as quickly. Namely, the main thread has a fixed stack size (which varies by platform). You cannot change it, in my experience. Other Threads have a default stack size, which again varies by platform, but you can set it to whatever you want.
Hello,
A RB/Xojo developer would not want to handle the physical and logical memory layout and its allocation and de-allocation on Windows, Linux and MAC OS alike for a data type such as “Integer” or “String”. Same should be the case for the stack size on Windows, Linux and MAC OS that is made available by RB/Xojo to the main thread. So if the Windows version of same source code does not cause “StackOverflow Exception” for say 52 recursive calls then the Linux version should also not cause a “StackOverflow Exception” for 52 recursive calls.
In other words, of what use is a claim that RB/Xojo can build application for Windows, Linux and MAC OS from same source code if the “Integer” data type on Windows (for example) can hold a value range of 0 to 65535 while on Linux the same “Integer” data type can only hold a value range of 0 to 255?
That is why it was mentioned that the application based on same RB/Xojo source code while processing same input data works fine on Windows while it causes “StackOverflow Exception” on Linux.
If any application requires very large stack size then most likely the details on this link can help .
I do not have a license, so will I be able to send feedback or a bug report based on the latest post of Joe Ranieri? If not then what could be the alternatives please apart from purchasing a license?
Thanks and Regards,
Shahid.
While recursion is often a natural solution to a problem it’s sometimes the least efficient way to actually solve the problem.
QuickSort is a great example of an algorithm that is very easy to describe recursively yet has a fairly straight forward iterative implementation that often operates faster & with less stack requirements than the recursive one.
And best of all YOU get to control how deep the recursion goes and can do something about it BEFORE you get a “Stack OverFlow Exception” as YOU manage the stack.
As for “how do I report a bug without owning a license” that’s a good question.