Cgi webapp throws unexplained exceptions

laurent_MIQUEL · January 14, 2020, 8:48am

Hello,

I’m having problems with a Xojo Webproject (CGI, xojo 2017r2.1) installed on an Ubuntu server 12.04.

Frequentely, the webapp does’nt respond or is very slow, we are forced to kill cgi process and relaunch webapp.

Below, see the syslog, these messages are repeated until the CGI crashes.
I see on webmin the use of the processor linked to the CGI gradually increases until reaching more than 90% of use and then cgi crashes.

We have no idea where this problem came from, or how to fix it.
What can we do to get more information about this problem (like what object throws the exception…) ?

Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV 2020-01-14 08:54:37 -> 0 Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebSession._CreateResponse%o<WebResponse>%o<WebSession> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.Respond%%o<WebPushHandler> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.CheckForChanges%%o<WebPushHandler> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.!Tickle Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 _SessionShutdownThread.Event_Run%%o<_SessionShutdownThread> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 /usr/lib/cgi-bin/cimserv/cimserv[4506]: last message repeated 3 times Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebSession._CreateResponse%o<WebResponse>%o<WebSession> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.Respond%%o<WebPushHandler> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.CheckForChanges%%o<WebPushHandler> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebPushHandler.!Tickle Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 _SessionShutdownThread.Event_Run%%o<_SessionShutdownThread> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV 2020-01-14 08:54:37 -> 2 Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 TextOutputStream.!Append%o<TextOutputStream>%o<FolderItem> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 cimserv.Event_UnhandledException%b%o<cimserv>o<RuntimeException> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebApplication.Event_UnhandledException%b%o<WebApplication>o<RuntimeException> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 /usr/lib/cgi-bin/cimserv/cimserv[4506]: last message repeated 2 times Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 _SessionShutdownThread.Event_Run%%o<_SessionShutdownThread> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 /usr/lib/cgi-bin/cimserv/cimserv[4506]: last message repeated 2 times Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 TextOutputStream.!Append%o<TextOutputStream>%o<FolderItem> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 cimserv.Event_UnhandledException%b%o<cimserv>o<RuntimeException> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 WebApplication.Event_UnhandledException%b%o<WebApplication>o<RuntimeException> Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 Jan 14 08:54:37 /usr/lib/cgi-bin/cimserv/cimserv[4506]: last message repeated 2 times Jan 14 08:54:37 ns355651 /usr/lib/cgi-bin/cimserv/cimserv[4506]: XOJO-CIMSERV ....2020-01-14 08:54:37 _SessionShutdownThread.Event_Run%%o<_SessionShutdownThread>

Regards

Greg_O_Lone · January 14, 2020, 8:56am

You should put some code in the Session and App UnhandledException events to save the exception stack to a log file. Thatll get you closer to the location.

Without more info, My experience with this tells me that youve either grown to the point where your server doesnt have enough memory for all of your simultaneous users, youve got a memory leak, or you have one or more runaway timers or tight loops which cause your server to kill the app.

DerkJ · January 14, 2020, 8:58am

Are you writing files? It may be blocking your application.
Be sure to do that in a thread.

As greg said it may be better to get the actual exception type using the App.UnhandledException and session.UnhhandledException events

Michel_Bujardet · January 14, 2020, 10:32am

I would send myself the stack by email in unhandledException, so I know where the bug is located in the program.