Windows shell very slow in calling DataAvailable event

I have an interesting problem that someone may be able to help with. I have an app that uses a shell to run a sub-process. I have a class that monitors the shell.DataAvailable event to get update information from stdout on the sub-process. On Mac this works a treat and I get new data quite frequently. On Windows, however, the event is very slow to trigger. For example I’ve only gotten called twice in 25 seconds, the first of which happens after about 18 seconds.

What decided when the DataAvailable event is triggered? Is there something I need to do in the C app to cause an update (perhaps some sort of flush instruction on stdout). There’s no Xojo setting that seems to be relevant.

It’s not that the Xojo app is busy doing anything else at this time. I start the shell in Asynchronous mode and then just wait for events to happen. CPU utilisation on the Xojo App is 0% during this part of the process and I’ve no code running. I’ve also tried changing the shell to Interactive mode with no difference.

Any suggestions welcome.

[quote=159840:@Ian Kennedy]I have an interesting problem that someone may be able to help with. I have an app that uses a shell to run a sub-process. I have a class that monitors the shell.DataAvailable event to get update information from stdout on the sub-process. On Mac this works a treat and I get new data quite frequently. On Windows, however, the event is very slow to trigger. For example I’ve only gotten called twice in 25 seconds, the first of which happens after about 18 seconds.

What decided when the DataAvailable event is triggered? Is there something I need to do in the C app to cause an update (perhaps some sort of flush instruction on stdout). There’s no Xojo setting that seems to be relevant.

It’s not that the Xojo app is busy doing anything else at this time. I start the shell in Asynchronous mode and then just wait for events to happen. CPU utilisation on the Xojo App is 0% during this part of the process and I’ve no code running. I’ve also tried changing the shell to Interactive mode with no difference.

Any suggestions welcome.[/quote]

Windows shell is notoriously slow, but 25 seconds seems outrageous, even for it. Are you sure your sub process runs normally ?

Yes, the application does work. It’s quite busy little app but if I run it from the Command Prompt it works a treat, outputting to the screen about once every second.

The flush method for stdout in c is…

fflush( stdout );

Should be worth a try

Thanks Chris. My outputs are using printf and include a
on the end of the line. Does not the
force a flush anyway? Worth a try I suppose.

If the app works fine in the Command Prompt, it simply means the DataAvailable is faulty. I am inclined to call that a bug, but one of the Xojo engineer may be able to bring some insight.

I remember Norman Palardy explaining in another thread that in Mac a shell is a child process, whereas it is not in Windows. That may explain the very loose handshake.

If it were not C, I would think about IPCSocket, but since it is not available, and stdout is not reported accurately by DataAvailable, why not instead have the C app send to a file the way you would with > in the command prompt ? Then you access it regularly through TextInputStream in sort of a poll. With some luck, the content will be accessible more regularly than through DataAvailable.

If for some reason the reroute of stdout did not update the file content right away, maybe you can create a series of small files each time you need to communicate with the app ?

I’m not sure files will suffice as we’re sometimes running multiple copies of the shell at the same time. I’m going to try the fflush suggestion. I’ve already tried dropping the DataAvailable event and simply reading shell.ReadAll in a timer, but that’s no better. I can but hope that the fflush solves the problem.

ps. If the fflush doesn’t solve it I will start a bug report. Providing something small and reproducible is perhaps an issue :slight_smile:

There must be something else.

I shelled to a batch file that reads :

echo %Time%
goto encore
[/code]

It outputs the time to stdout in a continuous flow.

I created a small program with

code s1.mode = 1
s1.execute “c:\users\mitch\desktop\infiniteloop.bat”[/code]

In DataAvailable, all I have is

textarea1.text = me.ReadAll+EndOfLine+textarea1.text

The update is permanent, no apparent lag of any sort. And certainly not 25 seconds.

I have a doubt, though. Are you sure the
actually generates the endofline expected by Windows to flush the buffer ? Does it show fine in the command prompt ?

Maybe the batch program works fine because it does terminate every line with CR/LF. You could try doing the same.

According to ISO it’s supposed to. However, I know if you redirect to a file then it doesn’t. Perhaps it’s not doing so when called from Xojo for similar reasons. As I say I’m going to add fflush(stdout); and test it. Will report the results back here.

When I run my helper app in a Command Prompt it works a treat. Continuous output as you would expect.

[quote=159865:@Ian Kennedy]According to ISO it’s supposed to. However, I know if you redirect to a file then it doesn’t. Perhaps it’s not doing so when called from Xojo for similar reasons. As I say I’m going to add fflush(stdout); and test it. Will report the results back here.

When I run my helper app in a Command Prompt it works a treat. Continuous output as you would expect.[/quote]

If the redirect to a file does not show returns, it is extremely possible that they lack the shell.

Do you get the proper CR/LF in your current DataAvailable (slow) event shell.ReadAll ? If fflush does not work, you may want to add &h0D+&h0A to your
and see if it solves the lag.

The file does show the returns, as does the ReadAll output. The redirect to a file just doesn’t flush when it meets a
. Sorry for not being clear.

[quote=159875:@Ian Kennedy]The file does show the returns, as does the ReadAll output. The redirect to a file just doesn’t flush when it meets a
. Sorry for not being clear.[/quote]

No need to be sorry. I did not understand the nuance.

If the fflush() does not do it, then I do not see why a simple batch file output gets updated several times a second in a mode 1 shell, and not a C program using stdout that shows up fine everywhere else. What could be the difference ?

I’m rather hoping for the fflush but I’m not at the machine with the C compiler today.

Another tip that I’ve discovered is that creating a timer that simply calls “YourShell.Poll” has greatly improved shell speed in Windows and Linux. Set the Timer period to 10 and then set the Timer mode to 2 when you start the backend process and set it to 0 when the task completes.

Since you have the C code…

you could make the C program talk over standard TCP sockets and ignore the shell all together.

or make the c program a DLL and call into it.

This might work as work around:

In this example you need to create:

  1. c:\test.bat
  2. PushButton1
  3. TexArea1
  4. Timer1 (Mode = Off)
  5. Public Property oExec in Window1 (oExec As OLEObject)

Content in c:\test.bat

dir c:\\

PushButton1.Action

  Dim WshShell  As OLEObject
  WshShell = NEW OLEObject("WScript.Shell")
  
  oExec = WshShell.Exec("c:\\test.bat")
  
  Timer1.Period = 100
  Timer1.Mode = timer.ModeSingle

Timer1.Action

  Dim OutText as String
  
  if Not oExec.StdOut.AtEndOfStream  Then 
    me.Period = 100 
    me.Mode = timer.ModeSingle
    
    OutText = OutText + oExec.StdOut.Read(255)
    
  end if
    
  TextArea1.AppendText( OutText) 
  

More info see: http://msdn.microsoft.com/en-us/library/cbxxzwb5(v=vs.84).aspx

[quote=159943:@John Hansen]This might work as work around:

In this example you need to create:

  1. c:\test.bat
  2. PushButton1
  3. TexArea1
  4. Timer1 (Mode = Off)
  5. Public Property oExec in Window1 (oExec As OLEObject)

Content in c:\test.bat

dir c:\\

PushButton1.Action

  Dim WshShell  As OLEObject
  WshShell = NEW OLEObject("WScript.Shell")
  
  oExec = WshShell.Exec("c:\\test.bat")
  
  Timer1.Period = 100
  Timer1.Mode = timer.ModeSingle

Timer1.Action

  Dim OutText as String
  
  if Not oExec.StdOut.AtEndOfStream  Then 
    me.Period = 100 
    me.Mode = timer.ModeSingle
    
    OutText = OutText + oExec.StdOut.Read(255)
    
  end if
    
  TextArea1.AppendText( OutText) 
  

More info see: http://msdn.microsoft.com/en-us/library/cbxxzwb5(v=vs.84).aspx[/quote]

I just found out there exist also a ReadAll method.

Change Timer1.Action to this:

    Dim OutText as String
  
  if Not oExec.StdOut.AtEndOfStream  Then 
    
    OutText = OutText + oExec.StdOut.ReadAll()
    
    if oExec.status <> 0 Then
      me.Period = 100
      me.Mode = timer.ModeSingle
    End If 
    
  end if
  
  TextArea1.AppendText( OutText) 
  

[quote=159852:@Ian Kennedy] Does not the
force a flush anyway? [/quote]
No.

Well a combination of adding fflush() calls and changing to Poll and ReadAll in a timer have made my app very responsive. Thanks for the suggestions.