Page 1 of 1

wftpserver service stops automatically // An error arose on

Posted: Tue Jan 08, 2013 3:14 pm
by eordona
Is anyone still having this error? We do, and here is the frequency (according to a log that I built starting in September 2012):

2012-09-12.18:00, wftpserver service restarted
2012-09-12.18:40, wftpserver service restarted
2012-10-25.09:55, wftpserver service restarted
2012-11-05.10:05, wftpserver service restarted
2012-11-08.15:40, wftpserver service restarted
2012-11-08.16:20, wftpserver service restarted
2012-11-08.17:00, wftpserver service restarted
2012-11-15.08:50, wftpserver service restarted
2012-11-23.11:05, wftpserver service restarted
2012-11-23.11:45, wftpserver service restarted
2012-11-23.12:25, wftpserver service restarted
2012-11-23.13:05, wftpserver service restarted
2012-12-07.12:15, wftpserver service restarted
2012-12-07.12:55, wftpserver service restarted
2012-12-14.10:10, wftpserver service restarted
2012-12-18.03:20, wftpserver service restarted
2012-12-18.04:00, wftpserver service restarted
2012-12-18.22:15, wftpserver service restarted
2012-12-19.13:50, wftpserver service restarted
2012-12-20.17:30, wftpserver service restarted
2013-01-03.15:40, wftpserver service restarted
2013-01-03.16:20, wftpserver service restarted
2013-01-04.14:55, wftpserver service restarted
2013-01-04.16:50, wftpserver service restarted
2013-01-05.09:15, wftpserver service restarted
2013-01-08.09:35, wftpserver service restarted

What shows in the "official" logs are LOTS of entries like this:

[14] Tue, 08 Jan 2013 09:34:34 An error arose on http port 443 (error code:24).
[14] Tue, 08 Jan 2013 09:34:34 An error arose on http port 443 (error code:24).
[14] Tue, 08 Jan 2013 09:34:34 An error arose on http port 443 (error code:24).
[14] Tue, 08 Jan 2013 09:34:34 An error arose on http port 443 (error code:24).

To combat this, I built a Linux shell script to run every 5 minutes, and if it detects this state it will restart the server. (I replaced my real domain name with mydomain.com to protect our anonymity:

#!/bin/sh

# uncomment for monitoring script results
#set -x

PROCESSDIR=/root/scripts
LOGFILE=$PROCESSDIR/wingftpmonitor.log
FAILLOG=$PROCESSDIR/wingftpmonitor.faillog
RUNTIME=`date +%Y-%m-%d.%H:%M`
TESTFILE=$PROCESSDIR/wingftpmonitor.tst
TESTFILE2=$PROCESSDIR/wingftpmonitor.tst2
LOGDIR=/root/wftpserver/Log/Domains/mydomain.com
LASTLOG=`ls -lrt $LOGDIR | tail -1 | awk {'print $9'}`

# check for error messages which indicates server not responsive
tail -10 $LOGDIR/$LASTLOG | egrep 'SSH initialize error|An error arose on' > $TESTFILE

touch $LOGFILE

# test the checkfile to see if it is bigger than 0 bytes
test -s $TESTFILE
if [[ $? == 0 ]]; then
# check one more thing, to see if wftpserver is running, we'll check the
# https login webpage. Since we are doing this from command line / lynx,
# we do expect to see the complaining message about requiring Javascript.
/usr/bin/lynx -dump https://mydomain.com" rel="nofollow" rel="nofollow" rel="nofollow" rel="nofollow | egrep 'The web client requires' > $TESTFILE2
test -s $TESTFILE2
if [[ $? -ne 0 ]]; then
# OK, server is not listening; restart it
/etc/init.d/wftpserver restart
# build a mailfile and mail it
echo "Subject: wftpserver service restart at $RUNTIME" >> $LOGFILE
echo "Listener error detected at $RUNTIME" >> $LOGFILE
echo " " >> $LOGFILE
echo "Unplanned restart of service occurred." >> $LOGFILE
echo " " >> $LOGFILE
echo "System may be restarted manually at command line:" >> $LOGFILE
echo " " >> $LOGFILE
echo " /etc/init.d/wftpserver restart " >> $LOGFILE
/usr/sbin/sendmail -v -t root < $LOGFILE
# add an entry into the faillog to keep track of recurring failures
echo "$RUNTIME, wftpserver service restarted" >> $FAILLOG
fi
rm $TESTFILE2
fi

rm $TESTFILE
rm $LOGFILE

This works, but it's not ideal. Bottom line is that something causes the WingFTP software to become unresponsive, killing availability until my workaround fixes it.

I am logging this as a bug.

Re: wftpserver service stops automatically // An error arose

Posted: Tue Jan 08, 2013 3:52 pm
by FTP
Did WingFTP service crash or just does not accept the connection?

The "error code 24" means there are too many open files, you can have a look on those articles for solving it:

http://stackoverflow.com/questions/2569 ... open-files" rel="nofollow
http://www.netadmintools.com/art295.html" rel="nofollow

Re: wftpserver service stops automatically // An error arose

Posted: Tue Jan 08, 2013 7:10 pm
by eordona
Thank you. I <presume> is crashes, but I'm not sure. When this happens I cannot connect to the server via SFTP, nor can I access the normal user web login via browser, nor the admin login via browser. Thanks for that info, I will research it.

Re: wftpserver service stops automatically // An error arose

Posted: Tue Jan 08, 2013 8:03 pm
by eordona
Based on your feedback and the URLs you sent, I made the following changes:

Edited file: /usr/include/linux/limits.h

Added the following line:
define NR_OPEN 65536

Edited: /etc/sysctl.conf

Added lines:
net.core.somaxconn=131072
fs.file-max=131072

Typed: sudo sysctl -p
... at the command line

Edited: /etc/security/limits.conf

Added lines:
* soft nofile 65535
* hard nofile 65535

rebooted.

ulimit -n used to return 1024, now it returns 65536.

Let's see if that fixes it.

Re: wftpserver service stops automatically // An error arose

Posted: Mon Feb 04, 2013 2:18 pm
by eordona
Nope, this is still failing. It has failed 10 times since my last post.

Re: wftpserver service stops automatically // An error arose

Posted: Mon Feb 04, 2013 2:32 pm
by FTP
Did you still get "error code:24"?

Re: wftpserver service stops automatically // An error arose

Posted: Mon Feb 04, 2013 3:29 pm
by eordona
Yes,
It looks like this happens when users are uploading files:

[05] Sun, 03 Feb 2013 22:05:45 (0000553) Start uploading file 'Copy of PDCA_Evansville Scrap_9_18_12.xls'
[05] Sun, 03 Feb 2013 22:05:47 (0000553) Start uploading file 'Dec. Scrap Dec 1 to Dec 16.MPJ'
[05] Sun, 03 Feb 2013 22:05:49 (0000553) Start uploading file 'EVV Scrap Summary_9_18_12.ppt'
[05] Sun, 03 Feb 2013 22:05:56 (0000553) Start uploading file 'EVV Scrap Summary_9_18_12A.ppt'
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).
[14] Sun, 03 Feb 2013 22:06:35 An error arose on http port 443 (error code:24).

Re: wftpserver service stops automatically // An error arose

Posted: Thu Oct 09, 2014 1:36 pm
by eordona
Just FYI. The 4.3.8 release has fixed this. On average my CentOS 6 based server was puking about 12 times a month, consistently. No exceptions. Since I have applied 4.3.8, the service has not failed once. So, whatever you did in 4.3.8 -- well done, this version is rock solid.

Re: wftpserver service stops automatically // An error arose

Posted: Mon Oct 13, 2014 2:48 am
by FTP
eordona wrote:Just FYI. The 4.3.8 release has fixed this. On average my CentOS 6 based server was puking about 12 times a month, consistently. No exceptions. Since I have applied 4.3.8, the service has not failed once. So, whatever you did in 4.3.8 -- well done, this version is rock solid.
Thanks for your feed back!