I'd like to see an FTP server upgrade

[Jeff]

Expanding on this thread http://aota.net/forums/showthread.php?t=23344 where I bemoaned the fact that the FutureQuest FTP server is 2.8x slower than a stock cpanel pure-ftpd server over my 1.5 / 256 satellite connection, I wanted to post an update (since that thread is now expired.)

I now have 3 Mbps / 512 kbps DSL here so latency is no longer an issue, and the FurtureQuest FTP server is still noticably slower than pure-ftpd. Before I thought it was just me since my latency to FutureQuest was ~1000 ms; now it's around 95 ms and still concurrent ftp sessions get the job done a lot faster to the other hosting servers I have that allow it. This is the only area I can think of where FutureQuest's hosting experience is noticeably inferior from my client perspective to an off-the-shelf offering (inferior meaning slower, taking more of my time to get the same task accomplished) so I just want to bemoan the fact again.

Secondly, today a client emailed that suddenly they were locked out of their FTP account after making no changes. It appears their FTP client didn't close the connection properly (or some other reason) and there is no way to quickly clear it for them nor is there any indication in the CNC of the block so suddenly the FTP is just "broken" to them. And they're locked out for at least 15 minutes while trying to get work done.

Overall I'd like to see the FTP server get an overhaul in the next round of upgrades. Somehow there has to be a way to bring it up to the standards of an off-the-shelf pure-ftpd server. Allow whitelisting client IPs in the CNC? Give more freedom if no login failures? Higher powered servers that can sustain more than 3 connections per client? I think the old ftp limits made sense in dialup one-file-at-a-time days, but now it's become a weak link that could be better (as we expect all things FutureQuest to be better than stock!) There has to be a way to keep the bots from overwhelming the ftp server without impacting legitimate clients noticeably.

[Bruce]

I now have 3 Mbps / 512 kbps DSL here so latency is no longer an issue, and the FurtureQuest FTP server is still noticably slower than pure-ftpd.We did a quick bandwidth test, and were able to transfer the same file at the same speed with both HTTP and FTP. As such, the FTP software itself is not the cause of any perceived slowness.

concurrent ftp sessions get the job done a lot faster to the other hosting servers I have that allow it. This is the only area I can think of where FutureQuest's hosting experience is noticeably inferior from my client perspective to an off-the-shelf offering (inferior meaning slower, taking more of my time to get the same task accomplished)Pragmatically, there is little difference between opening a pile of connections to transfer the same large file and a small scale denial-of-service attack. As both the servers and the bandwidth is shared with other site owners, we cannot allow any one site to significantly damage the quality of service for all the others. Simply put, the limit on the number of concurrent sessions is an administrative policy put into place to protect all site owners against resource starvation by others.

Secondly, today a client emailed that suddenly they were locked out of their FTP account after making no changes. It appears their FTP client didn't close the connection properly (or some other reason) and there is no way to quickly clear it for them nor is there any indication in the CNC of the block so suddenly the FTP is just "broken" to them. And they're locked out for at least 15 minutes while trying to get work done.I agree this is a deficiency in the current implementation. I have actually looked into correcting this in the past week, after correcting the 2GB file limitation (http://www.aota.net/forums/showthread.php?postid=169204#post169204). I will bump it up the priority list.

I think the old ftp limits made sense in dialup one-file-at-a-time days...If anything, it makes more sense now to have the limits than it did when the potential bandwidth to FTP clients was more constrained, precisely because there is more opportunity for abuse.

[Jeff]

We did a quick bandwidth test, and were able to transfer the same file at the same speed with both HTTP and FTP. As such, the FTP software itself is not the cause of any perceived slowness.
I think it may be the transfer of many small files vs. one big one that highlights the advantage of concurrent connections; even with 100 ms of latency only now, I think with many small files I spend more time waiting for the go-back-and-forth where concurrent saturates my upload speed nearer 100% of the time.) I am in a remote location, so maybe this is unique to me even though the two ISPs in a row are different technology.
Pragmatically, there is little difference between opening a pile of connections to transfer the same large file and a small scale denial-of-service attack. As both the servers and the bandwidth is shared with other site owners, we cannot allow any one site to significantly damage the quality of service for all the others. Simply put, the limit on the number of concurrent sessions is an administrative policy put into place to protect all site owners against resource starvation by others.
The difference in my mind is that I'm a paying client and the current administrative policy makes your service slower--Would 8 connections x the number of customers typically ftping updates to their site at a given time really overwhelm the FTP server? What would it take to increase the resources available to the FTP server so it could handle such use without issue?
I realize I would be crying even more loudly if bots took up all the available ftp slots causing a much worse issue. My question is what would it take to increase the number of connection slots available so more could be available to legitimate paying customers to speed things up? Is it the core linux OS that limits the total number of FTP slots available? Could faster hardware support more FTP slots in the future? How about smarter software that allows you to enter a legitimate paying client IP range in the CNC and allow those more ftp slots, thus limiting DOS bots as much as possible but allowing paying clients ample resources to get the job done as fast as possible?
Secondly, today a client emailed that suddenly they were locked out of their FTP accountI agree this is a deficiency in the current implementation. I have actually looked into correcting this in the past week, after correcting the 2GB file limitation. I will bump it up the priority list.
Fantastic -- thank you, I appreciate it -- I really don't like explaining to a client that they'll have to wait 15 minutes to be able to get their work done and there is nothing I can do to clear it. While I'd like to see you look to see if the FTP server can be improved in the future RoadMap, this is the immediate thing that causes me frustration.
If anything, it makes more sense now to have the limits than it did when the potential bandwidth to FTP clients was more constrained, precisely because there is more opportunity for abuse.I appreciate your concern for the stability and quantity of service. Still this is one area that doesn't work well for me. It's extremely rare for FutureQuest services to be the slower, limiting factor for me.

[Jeff]

P.S. I don't mean my message above to sound ornery (though I was unhappy today as I don't like lacking the ability to help a client connect to their hosting account) It's not that I don't appreciate everything you all do to keep things running smoothly. It does irk me a bit to connect to my "slower" ftp account here, but I can live with it as it's certainly plenty workable. Still I hope it can be made superior to others in the Future like other things FutureQuest.

Here is my current connection which is the best that has every been available on the Island here, though certainly not nearly as fast as what urban areas have now:

3 48 ms 47 ms 38 ms mdtnwiallcr52-v28.network.tds.net [69.128.254.145]
4 51 ms 57 ms 56 ms xo.peering.tds.net [64.50.236.246]
5 59 ms 58 ms 52 ms 207.88.12.33.ptr.us.xo.net [207.88.12.33]
6 102 ms 108 ms 102 ms 207.88.12.34.ptr.us.xo.net [207.88.12.34]
7 105 ms 100 ms 109 ms te-3-0-0.rar3.atlanta-ga.us.xo.net [207.88.12.9]
8 99 ms 105 ms 101 ms te-3-2-0.rar3.miami-fl.us.xo.net [207.88.14.57]
9 103 ms 98 ms 106 ms 207.88.14.58.ptr.us.xo.net [207.88.14.58]
10 90 ms 91 ms 90 ms w026.z207088246.xo.cnc.net [207.88.246.26]
11 95 ms 87 ms 94 ms border4.ge3-0.bbnet2.mia003.pnap.net [69.25.0.76]
12 99 ms 109 ms 110 ms futurequest-2.border3.mia003.pnap.net [216.52.162.126]
13 92 ms 96 ms 93 ms futurequest.mtld.twtelecom.net [66.194.200.14]
14 87 ms 92 ms 97 ms p.core-x-04.futurequest.net [69.5.31.237]
15 96 ms 96 ms 95 ms futurequest.net [69.5.6.116]

Pinging futurequest.net [69.5.6.116] with 32 bytes of data:
Reply from 69.5.6.116: bytes=32 time=93ms TTL=56
Reply from 69.5.6.116: bytes=32 time=95ms TTL=56
Reply from 69.5.6.116: bytes=32 time=94ms TTL=56
Reply from 69.5.6.116: bytes=32 time=108ms TTL=56
Reply from 69.5.6.116: bytes=32 time=97ms TTL=56
Reply from 69.5.6.116: bytes=32 time=92ms TTL=56
Reply from 69.5.6.116: bytes=32 time=98ms TTL=56
Reply from 69.5.6.116: bytes=32 time=90ms TTL=56
Reply from 69.5.6.116: bytes=32 time=86ms TTL=56
Reply from 69.5.6.116: bytes=32 time=103ms TTL=56
Reply from 69.5.6.116: bytes=32 time=93ms TTL=56
Reply from 69.5.6.116: bytes=32 time=87ms TTL=56
Reply from 69.5.6.116: bytes=32 time=92ms TTL=56
Reply from 69.5.6.116: bytes=32 time=88ms TTL=56
Reply from 69.5.6.116: bytes=32 time=95ms TTL=56
Reply from 69.5.6.116: bytes=32 time=86ms TTL=56
Reply from 69.5.6.116: bytes=32 time=95ms TTL=56
Reply from 69.5.6.116: bytes=32 time=96ms TTL=56
Reply from 69.5.6.116: bytes=32 time=99ms TTL=56
Reply from 69.5.6.116: bytes=32 time=97ms TTL=56

Ping statistics for 69.5.6.116:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 86ms, Maximum = 108ms, Average = 94ms

I did the following test:
Files: 36
Folders: 6

Time to Pure-FTPD server with 4 concurrent transfers
24 seconds

Time to Pure-FTPD server with 8 concurrent transfers
18 seconds

Time to FutureQuest FTP server with 1 concurrent transfer
38 seconds

Time to FutureQuest FTP server with 2 concurrent transfers
35 seconds

These are not very scientific tests -- just starting the transfer with FileZilla on the minute and then noting the windows clock when finished.

It does appear that my other provider is slightly closer to me, so the 5ms difference in latency likely contributes as well, though I suspect only a small amount.
Minimum = 78ms, Maximum = 105ms, Average = 89ms

[Bruce]

Secondly, today a client emailed that suddenly they were locked out of their FTP account after making no changes. It appears their FTP client didn't close the connection properly (or some other reason) and there is no way to quickly clear it for them nor is there any indication in the CNC of the block so suddenly the FTP is just "broken" to them.The FTP server has been upgraded today specifically to fix this concern. It should now be much more responsive to dropped connections, properly terminating the session as appropriate. :yeah:

[Jeff]

Excellent - thanks Bruce.