Passing parameters to Python script with Shell.Execute

  1. ‹ Older
  2. last year

    Markus W

    10 Jun 2018 Pre-Release Testers #JeSuisHuman New Zealand, Auc...

    Why do you want to manipulate the strings in Python instead of in Xojo? The only reason I could see is for example to use BioPython, but in that case ASCII should do fine …

  3. Robert L

    10 Jun 2018 Federal Way, WA (Seattle Area)

    Why do you want to manipulate the strings in Python instead of in Xojo?

    Short answer. It is more than strings.

    Intermediate length answer :)

    Xojo and Python are very different places (compiled vs interpreted; commercial vs open source; obscure vs hugely popular; small number of third party resources vs huge libraries of third party add-ons; easy creation of graphical user interface vs difficult and immature attempts)

    I am really trying to create the infrastructure that would allow me, when working on projects in Xojo, to drop down into Python and perform certain tasks in that environment and then return the results to Xojo for display or further refinement. My hope is that if I can overcome/understand all the glitches here, and figure out the process, then going forward I will not have to worry about the issue.

    I am an intermediate Xojo programmer and a beginning Python programmer. I like creating apps for my own use and in some cases offer them for free to others. But I am not a commercial developer so paying $ for third party add-ons to Xojo is onerous for me. The extensive library add-ons of Python are free.

    I generally prefer writing my code in Xojo but even as a beginning Python programmer I see certain approaches to problems are easier for me in Python. My biggest problem in Python is not being able to create applications with nice GUI interfaces which I can do in Xojo.

    I am not solely interested in processing strings in Python. It so happens that strings are the only way I know (apart from writing files to disk to access from both environments) to pass data back and forth between Xojo and Python. Changing strings back numbers or arrays or whatever is not hard in either environment. So the string emphasis is just an artifact of trying to come up with a smooth way of pushing data back and forth.

    There are circumstances that in fact I do want to process strings in Python rather than in Xojo. You mention BioPython as an example. I find working with Regex in Python (with re library) is easier for me than the same in Xojo. One thing that launched me in this direction of trying to learn Xojo/Python intercommunication was an occasion that I wanted to use Levenshtein distance as a tool to find name matches in a context with a couple thousand name and some of the names were misspelled. Implementing Levenshtein distance in Xojo code is complicated for me, and I see in Python that I can find a library that will supply this code without my having to understand it and actually implement it myself.

    NumPy & SciPy supply all these tools for scientific computing.

  4. Markus W

    10 Jun 2018 Pre-Release Testers #JeSuisHuman New Zealand, Auc...

    @RobertLivingston I am an intermediate Xojo programmer and a beginning Python programmer.

    Same here.

    My biggest problem in Python is not being able to create applications with nice GUI interfaces which I can do in Xojo.

    Same here.

    If Python got a decent compiler and IDE it would wipe the floor with anything else …

    One thing that launched me in this direction of trying to learn Xojo/Python intercommunication was an occasion that I wanted to use Levenshtein distance as a tool to find name matches in a context with a couple thousand name and some of the names were misspelled..

    How about using an [in-memory?] database and doing a LIKE search instead?

  5. Beatrix W

    10 Jun 2018 Pre-Release Testers, Third Party Store Europe (Germany)

    Once upon a time I felt the lure of Python, too. So many many nice codes. Alas, Xojo as language is 1000% better than Python.

    1. whitespace (majorly awful)
    2. case sensitive (are they nuts?)
    3. multiple returns from a function

    The latter one broke me as I never could be sure what a function would return. Commercial code had to make sure that everything was in place - stuff that in Xojo the compiler enforces.

    For a private project: more fun to you.

    Oh, the IDE I used - PyCharm - was pretty nifty. Way too small buttons. But really nice to use. Actually working auto-complete. Debugger values shown inline with the code - that was great.

  6. Markus W

    10 Jun 2018 Pre-Release Testers #JeSuisHuman New Zealand, Auc...

    @Markus W wanted to use Levenshtein distance as a tool to find name matches in a context with a couple thousand name and some of the names were misspelled.

    P.S. That would also be VERY slow in my opinion …

  7. Robert L

    10 Jun 2018 Federal Way, WA (Seattle Area)

    I just used Levenshtein distance as an example. I did implement what I needed in yet another language, and it was extremely slow and I had to "translate it" into that language from whatever I could find on the web and debug the code etc. I am happier if I can find a pre-baked solution.

    I have not used the Python implementation so I have no first hand experience. But my impression is that the Python implementation is done as a call to complied C code which was written by professionals to be fast and efficient. Once I get my Xojo/Python linkage project worked out, I will try it.

    As for the slowness, it was very slow but I used various "cheats" like filtering out cases that neither the first letter of the last or first names matched and in this way made it more manageable.

  8. Paul F

    11 Jun 2018 Pre-Release Testers, Xojo Pro Europe, Cambridge

    You may already know of this but thought I'd mention that Joe Strout has a String Utils module on his website that calculates Levenshtein distance along with loads of other functions.

  9. @RobertLivingston My biggest problem in Python is not being able to create applications with nice GUI interfaces which I can do in Xojo.

    wxPython uses the native widgets of each supported platform.

    For an IDE you could look at Visual Studio Code

    I think Einhugur offer some classes to enable interaction between Xojo and Python.

  10. Markus W

    11 Jun 2018 Pre-Release Testers #JeSuisHuman New Zealand, Auc...

    http://www.strout.net/info/coding/rb/intro.html

  11. Robert L

    11 Jun 2018 Federal Way, WA (Seattle Area)

    wxPython is a cross-platform GUI toolkit for the Python programming language. It allows Python programmers to create programs with a robust, highly functional graphical user interface, simply and easily.

    While I am very glad that wxPython and the Phoenix variant exist, when compared with using Xojo, these are a long way from easy IMO.

    I have just started working with my daughter on a QGIS project. QGIS Amazing free resource if you are into cartography. But it is one of those things that is married to Python. I would like, in my own life, to get the best of Xojo and Python. So I could do something like working with Xojo linked with Python to do QGIS stuff.

    I will say no more until I have actually done something interesting. Talk/speculation is cheap.

    And I have hijacked my own thread. :(

  12. Robert L

    11 Jun 2018 Federal Way, WA (Seattle Area)

    Summary:
    The problem originally described was one of reliably passing data in the form of string(s) to a Python script using Shell.Execute. There is the related issue of getting data back from that same Python script using Shell.Result.

    Const SPACE = " "
    Const PATH_PYTHON3 = "/Users/owl/anaconda3/bin/python"
    Const PATH_PYTHON_SCRIPT = "/Users/owl/Documents/PythonPractice/PythonLittleApps/Trial.py"
    
    Dim argument1 As String
    argument1 = "abcde"
    Dim argument2 As String
    argument2 = "pqrst"
    
    Dim pyShell As New Shell
    pyShell.Execute(PATH_PYTHON3 + SPACE + PATH_PYTHON_SCRIPT + SPACE + argument1 + SPACE + argument2)
    
    Dim sResult As String
    sResult = pyShell.Result
    MsgBox(sResult)

    One difficulty is passing certain characters that are misinterpreted when they go to the Shell. Characters like $; !; < etc.

    If you need to pass such characters, one can figure out what all these characters are and escape them before sending them to the Shell

    Tim Jones basically contributed:

    Const METHOD_NAME As Text = "FixSpecial"
    Const CHARACTERS_THAT_MUST_BE_ESCAPED As String = "\ ` ~ ! @ # $ % ^ & * ( ) | { } [ ] ' ; < > ? """
    For Each iChar As String In Split(CHARACTERS_THAT_MUST_BE_ESCAPED, " ") // It is split on space.
      sEscaped = sEscaped.ReplaceAll(iChar, "\" + iChar) // each trouble character gets replaced with escaped version
    Next
    
    sEscaped = sEscaped.ReplaceAll(" ", "\ ") // spaces get escaped
    
    Return sEscaped

    Markus Winter noted that if you enclose the arguments in quotes most of the troublesome characters do not need to be escaped although a few still do and based on this:

    Const METHOD_NAME As Text = "EscapedArgument"
    Const SQ = &u27 // ' Single Quote
    Const DQ = &u22 // " Double Quote
    
    Dim backslashAndDollarSign As String = "\" + "$"
    Dim backslashAndBacktick As String = "\" + "`"
    Dim backslashAndSingleQuote As String = "\" + SQ
    Dim backslashAndDoubleQuote As String = "\" + DQ
    
    arg = ReplaceAll(arg, "`", backslashAndBacktick)
    arg = ReplaceAll(arg, "$", backslashAndDollarSign)
    arg = ReplaceAll(arg, SQ, backslashAndSingleQuote)
    arg = ReplaceAll(arg, DQ, backslashAndDoubleQuote)
    
    arg = DQ + arg + DQ
    
    Return arg

    Either of these formulation work equally well as best I can tell. Just run the arguments through one or the other of these functions before passing them to pyShell.Execute

    pyShell.Execute(PATH_PYTHON3 + SPACE + PATH_PYTHON_SCRIPT + SPACE + FixSpecial(argument1) + SPACE + EscapedArgument(argument2))

    The Trial.py script might be:

    import sys
    def main():
    
        passedArg1=sys.argv[1]
        passedArg2=sys.argv[2]
        sReverseArg1 = PassedArg1[::-1]
        print(sReverseArg1)
        print(passedArg2)
    
    main()

    This suffices for when you want to accept and send back simple ASCII text.

    The next problem was sending and receiving back non-Ascii characters should you need to do that. Characters like é. Basically, the above does not work.

    It turns out that easiest solution really lies on the Python side of things.

    If you want to deal with Non-Ascii characters either when you receive them or send them back from Python when called from the Shell.Execute, try adding the codec "stuff" to the PYTHON_SCRIPT code

    import sys
    def main():
    
        import codecs
        sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
    
        passedArg1=sys.argv[1]
        passedArg2=sys.argv[2]
        sReverseArg1 = PassedArg1[::-1]
        print(sReverseArg1)
        print(passedArg2)
    
    main()

    And then it works.

    **********

    P.S. Beatrix suggested the possibility of using BASE64 encoding to traverse the gap from Xojo to Python and then Python back to Xojo.

    BASE64 encoding represents the text in characters that are all part of the ASCII set and which contain none of the troublesome characters like $ and <. This methodology can also deal with non-ASCII characters because they get coded as BASE64 characters that are all ASCII.

    I submit this here just in case some situation arises that this approach is preferable. I did get it to work although not very elegant.

    Const SPACE = " "
    Const PATH_PYTHON3 = "/Users/owl/anaconda3/bin/python"
    Const PATH_PYTHON_SCRIPT = "/Users/owl/Documents/PythonPractice/PythonLittleApps/TrialB64.py"
    Const NO_LINE_BREAK = 0
    
    Dim argument1 As String
    argument1 = "abcde"
    Dim argument2 As String
    argument2 = "pqrst"
    
    Dim pyShell As New Shell
    pyShell.Execute(PATH_PYTHON3 + SPACE + PATH_PYTHON_SCRIPT + SPACE + EncodeBase64(argument1, NO_LINE_BREAK) + SPACE + EncodeBase64(argument1, NO_LINE_BREAK))
    
    Dim sResult As String
    sResult = DecodeBase64(pyShell.Result, Encodings.UTF8)
    MsgBox(sResult)

    On the Python side of things, you do not have to use codec "stuff" because relying on BASE64 encoding and decoding.

    import sys
    def main():
        import os
        import base64
        # decode to get the string that was passed
        incomingBase64Arg1 = sys.argv[1] # gets the Base64 from Xojo
        bDecodedArg1 = base64.b64decode(incomingBase64Arg1) # get the binary string
        utf8DecodedArg1 = bDecodedArg1.decode('utf-8') # convert to a utf8 string
    
        # do something in Python (reverse for example)
        sReverseArg1 = utf8DecodedArg1[::-1]#+"<CR>"#+"\n"
    
        # code in preparation for sending it out from Python to Xojo
        outgoingBase64Arg1=base64.b64encode(bytes(sReverseArg1,'utf-8')).decode('utf-8')
    
        # Put in output buffer
        print(outgoingBase64Arg1)
    main()

    I apologize for any typo's or stupid things that may exist in the code samples.

  13. Tim J

    11 Jun 2018 Pre-Release Testers, Xojo Pro Dehydrating in AZ

    The Tkinter occam that ties the TCL/TK UI components into Python is your best option for truly cross platform support - every Unix, Mac OS, and Windows supports it. You could always look into PAGE for building a Tkinter GUI for Python use. While not as directly tied in like JavaFX Scene Builder or Xojo, it's still pretty good at giving you what you need (and enforcing a MVC model design).

    PAGE Editor on Sourceforge

  14. Robert L

    11 Jun 2018 Federal Way, WA (Seattle Area)

    Thanks. But my intent is to use Xojo for lots of the coding work and not only the GUI. So I can use Python if I need it to crack part of a problem and Xojo to crack other parts. And make the GUI in Xojo.
    Fortunately, since this is all for myself, I do not have to worry about dual platform.
    But dual platform is certainly interesting in that Xojo CAN do it but injects a little more complexity with edge stuff.
    Meanwhile, I will look at some of these Python GUI builders. But I do not see one that everybody says this is great and easy.

  15. 9 months ago

    Robert L

    18 Jan 2019 Federal Way, WA (Seattle Area)

    I provided some sample code above and below is a little snippet from that submission.

    passedArg1=sys.argv[1]
    passedArg2=sys.argv[2]
    sReverseArg1 = PassedArg1[::-1]
    print(sReverseArg1)
    print(passedArg2)

    Python is case-sensitive so there is a bug here. The capital "P" in PassedArg1[::-1] is wrong. It should be passedArg1[::-1]

    sReverseArg1 = passedArg1[::-1]

    Just in case some tries copying and pasting this code and gets an error message as I did today.

  16. Oliver O

    19 Jan 2019 Pre-Release Testers, Xojo Pro https://udemy.seminar.pro

    Did you check out the python plugins of Björn?
    http://einhugur.com/Html/ScriptEngines.html

  17. Robert L

    20 Jan 2019 Federal Way, WA (Seattle Area)

    Thanks for the heads up. I haven't, and I should look at the demo.
    But I am a non-commercial, hobbyist programer and avoid plugins for this reason.
    I like Python and am curious about it. And I wonder just how much I can do with just using what is built in to Xojo. My "experiments" are early, but intriguing. My interests focus around:

    1. Using Xojo as a GUI for Python
    2. Using Python to accomplish certain tasks that are not easy to do in Xojo. So in this scenario, Python is acting as a silent supplement to Xojo.

    The key to doing this is figuring out how to pass data back and forth between the two environments. As my use expands, I find myself writing little reusable bits of code in Python and Xojo to make it easier to communicate between the two. I do not know how far this will go. It is a typical hobbyist project with a taint of tilting at windmills. But I have one solid "working" solution to another wise annoying problem which was enabled by marrying Xojo with Python.
    I wonder if Xojo itself could embrace being a GUI for Python. There is a need for Python to have a reasonable GUI tool. What they have to date, doesn't seem to me to be very inspiring.

  18. @RobertLivingston I wonder if Xojo itself could embrace being a GUI for Python. There is a need for Python to have a reasonable GUI tool.

    I don't think there's much mileage in using Xojo for that, Python has plenty of high quality options for creating a GUI:

    wxPython (Native)
    Qt for Python (Qt)
    PyGObject (GTK)
    Kivy (Kivy)
    PySciter (Sciter)
    IronPython (.NET)

  19. Robert L

    20 Jan 2019 Federal Way, WA (Seattle Area)

    Thanks for this list and I should look into these more. But just the fact that there are 6 choices makes me think that none of them can be that great.
    And to my knowledge, all of these are specifying the GUI in code. In these Python GUI creation tools, you are writing out code to specify the size and location of a Button for example. I remember doing this type of programming in the 1990's using Foxbase. And I am not eager to return to that methodology.
    In Xojo, if you want a Button of such and such size in a location, you place there in the IDE and size it in the IDE without writing actual code. Xojo takes care of it. It is a far different experience.
    The fact that I can write a Python program with a GUI using these listed tools is fine. But for me, not attractive.

  20. Tim J

    20 Jan 2019 Pre-Release Testers, Xojo Pro Dehydrating in AZ

    Follow my link to PAGE above. It is a GUI layout editor. Also, take a look at the "Getting Started" doc.

    Many of us older REALbasic users will recognize the designer layout :).

  21. @RobertLivingston Thanks for this list and I should look into these more. But just the fact that there are 6 choices makes me think that none of them can be that great.

    There are six different modules because they are for six different GUI toolkits. I've only listed the ones that are great, I could have included PyFLTK, TKinter and others but they're not so good (imo).

    wxPython is the one most similar to Xojo in that it uses the platform's native widgets.
    .

    @Robert Livingston And to my knowledge, all of these are specifying the GUI in code. In these Python GUI creation tools, you are writing out code to specify the size and location of a Button for example.

    That's optional. wxPython, Qt for Python, PyGObject and ironPython all have drag and drop visual designers.

or Sign Up to reply!