[Release] ProFTPD (FTP server) for VMware ESXi 5.x

ProFTPD is a GPL-licensed, feature rich and highly configurable FTP server. Today I have made available an Offline Bundle to install ProFTPD in VMware ESXi 5.x. Installing it will enable you to transfer files from and to an ESXi host using an FTP client program.

Download

Installation

Upload the Offline Bundle to a datastore of your host using the vSphere client's datastore browser. Then run the following command in a local or remote ESXi shell to install the bundle:
esxcli software vib install --no-sig-check -d /vmfs/volumes/datastore1/ProFTPD-1.3.3-8-offline_bundle.zip
(replace datastore1 with the name of your datastore)

You do not need to put the host in maintenance mode before, and you do not need to reboot the host after the installation!

To uninstall the package run the following command
esxcli software vib remove -n ProFTPD
Uninstallation also does not require maintenance mode or rebooting.

Please read this post for more information about installing unsigned software packages in ESXi.

Features
  • Supports active and passive FTP
  • Fully manageable in the vSphere client (like the builtin system services): Use the Configuration / Security Profile dialog to start, stop or restart the service, to set the startup mode and enable/disable associated firewall rules

      

  • The commented config file /etc/proftpd.conf is editable, changes will persist upon reboots!
  • Logging is disabled by default (to not fill the RAM disk). Can be changed in /etc/proftpd.conf.

Compatibility

ProFTPD runs fine in ESXi 5.1 and offers good transfer performance. You will notice messages in /var/log/vmkernel.log about  the TCP Socket option SO_OOBINLINE not being available, but this can be safely ignored.

It will also work on ESXi 5.0, but here the VMkernel.log file fills up with error messages about unimplemented Linux functions. I recommend upgrading to ESXi 5.1 before using the package.

Background

I personally do not see much value in having an FTP server in ESXi, because - if you really need to occasionally transfer files directly from/to ESXi - then you can also use other methods like the vSphere client's datastore browser or Veeam's free FastSCP tool.

I created this package purely for self-educational purposes: I wanted to find out how to create an ESXi installation package for a network service, and I will soon share what I have learnt in one or two more blog posts. Stay tuned!

65 comments:

  1. Not installed.
    ESXi 5.1.0, 838463

    ~ # esxcli software vib install --no-sig-check -d /opt/ProFTPD-1.3.3-8-offline
    _bundle.zip
    [InstallationError]
    The following services were not started: /etc/init.d/proftpd
    It is not safe to continue. Please reboot the host immediately to discard the unfinished update.

    ----------------------------------------------------
    ESXi 5.0.0, 721882
    ~ # esxcli software vib install --no-sig-check -d /opt/ProFTPD-1.3.3-8-offline_bundle.zip
    [DependencyError]
    VIB VFrontDe_bootbank_ProFTPD_1.3.3-8's acceptance level is accepted, which is not compliant with the ImageProfile acceptance level certified
    To change the host acceptance level, use the 'esxcli software acceptance set' command.
    ----------------------------------------------
    really need this bundle. Please help me.

    ReplyDelete
  2. Hi Alex,
    For the first case please send me the /var/log/esxupdate.log file via mail to [email protected].
    For the second case lower the system's acceptance level like this:
    esxcli software acceptance set --level=VMwareAccepted
    - Andreas

    ReplyDelete
  3. Thanks a lot. On esxi 5.0 installed and running. Sending logs from esxi 5.1.

    ReplyDelete
  4. I'm getting the [InstallationError] as described by Alex above. Any resolution yet?

    ReplyDelete
    Replies
    1. Andreas - I resolved the problem. I was trying to install your package on a freshly installed ESXi 5.0.0. The host had a DHCP address, but I had not configured the host name. I hacked it with a quick entry in /etc/hosts and all is well now. Thank you for this great package; makes life a LOT easier.

      Here are the log entries that tipped me off to the problem.

      2013-01-19T05:23:26Z esxupdate: LiveImageInstaller: DEBUG: Running [['/etc/init.d/proftpd', 'start', 'install']]...
      2013-01-19T05:23:26Z esxupdate: vmware.runcommand: INFO: runcommand called with: args = '['/etc/init.d/proftpd', 'start', 'install']', outfile = 'None', returnoutput = 'True', timeout = '0.0'.
      2013-01-19T05:23:26Z esxupdate: LiveImageInstaller: DEBUG: output: Starting proftpd
      - warning: unable to determine IP address of 'localhost.mchan.com'
      - error: no valid servers configured
      - Fatal: error processing configuration file '/etc/proftpd.conf'
      2013-01-19T05:23:26Z esxupdate: LiveImageInstaller: WARNING: Error in starting service '/etc/init.d/proftpd': ([], "Error in running ['/etc/init.d/proftpd', 'start', 'install']:\nReturn code: 1\nOutput: Starting proftpd\n - warning: unable to determine IP address of 'localhost.mchan.com'\n - error: no valid servers configured\n - Fatal: error processing configuration file '/etc/proftpd.conf'\n\nIt is not safe to continue. Please reboot the host immediately to discard the unfinished update.")

      Delete
    2. Great, many thanks for your feedback!

      Delete
  5. Using ProFTPD I was able to transfer a 100GB VMDK file in little over 2 hours to a local datastore on an ESXi 5.1 box. Using the upload feature in the built-in Datastore browser, it would have cost me at least 48 hours to transfer the same file from the same source to the same destination.

    So there is a lot of value in your FTP package, you're the first one to implement this, and offer a solution for the terrible implementation of native upload mechanisms available in the vSphere client.

    Thanks a lot!

    Joris D

    ReplyDelete
    Replies
    1. Something must be wrong on your side then, I can transfer files in and out via Datastore browser with the maximum possible speed (limited only by given datastore and network performance, whichever is slower).
      Only thing that comes to my mind (except simply having extremely slow hardware) is a traffic shaping set on management traffic.

      Delete
  6. Hello,
    I've this error when I try to install it:
    [VibDownloadError]
    Failed to download VIB.
    url = ProFTPD-1.3.3-8
    localfile = Unable to download VIB from any of the URLs
    Please refer to the log file for more details.

    Thanks and regards, Gustavo

    ReplyDelete
    Replies
    1. Gustavo,

      what command line did you use? Have you uploaded the Offline Bundle to a datastore of the host and referenced this file in the esxcli command?

      Delete
    2. Hello, the command was:
      esxcli software vib install --no-sig-check -d /vmfs/volumes/WD/ftp/ProFTPD-1.3.3-8-
      offline_bundle.zip

      I've put ProFTPD-1.3.3-8-
      offline_bundle.zip on /vmfs/volumes/WD/ftp

      Thsnks again!! Gustavo

      Delete
    3. /vmfs/volumes/516f8a35-9757b074-21bb-00237d9ca887/ftp # ls -la
      drwxr-xr-x 1 root root 420 May 22 14:04 .
      drwxr-xr-t 1 root root 1540 May 22 13:38 ..
      -rw------- 1 root root 233858 May 22 14:04 ProFTPD-1.3.3-8-offline_bundle.zip
      /vmfs/volumes/516f8a35-9757b074-21bb-00237d9ca887/ftp # pwd
      /vmfs/volumes/WD/ftp

      Delete
    4. Gustavo, just one other thought: Make sure that you do not have any space characters in the file name, and try to enclose the file name in double quotes ( ... -d "/vmfs/...")

      Delete
  7. I've tried with double quotes and the same error. I've sent you the log by email!Thanks, Gustavo

    ReplyDelete
    Replies
    1. After some quick mail exchange and looking at esxupdate.log it turned out that gustavo ran into the same error as Alex Kozlov and Sam McHan (see comments above).
      The resolution was to add an entry to /etc/hosts.

      Delete
    2. It worked! Thanks a lot!

      Masking some test, the speed es similar comparing ftp and downloading from the vshere browser. Some users had told that with fpt the speed will increase. Not in my case :(

      Delete
    3. The FTP protocol has surely less overhead than the NFC protocol used in vSphere, but that does not help if your throughput is limited by other factors. What about disk and network i/o?

      Delete
  8. It works. Very useful program. Save tons of hours. Thank you so very much for sharing.

    ReplyDelete
  9. This package is great. Only issue is FTP security but that can be handled from other angles

    good work.

    ReplyDelete
  10. Im getting timed out errors when trying to connect to the server from filezilla client...

    Error: Connection timed out
    Error: Could not connect to server

    I can see that proftpd is running and the port 21 is open..

    Can you help me?

    ReplyDelete
    Replies
    1. Can you ping the host?
      Can you connect via the vSphere Client from this computer?
      Doublecheck if the service is running and the firewall port open.
      Doublecheck the host name resp. IP address that you use in Filezilla.
      Can you telnet to port 21 of the host?
      Do you have a firewall on your local computer that may block outgoing FTP?

      Delete
  11. I am getting the following error when installing the package:
    2013-08-13T14:28:43Z esxupdate: root: ERROR: MetadataDownloadError: ('zip:/vmfs/volumes/datastore1/utilities/ProFTPD-1/ProFTPD-1.3.3-8-offline_bundle.zip?index.xml', '', 'Could not download from depot at zip:/vmfs/volumes/datastore1/utilities/ProFTPD-1/ProFTPD-1.3.3-8-offline_bundle.zip?index.xml, skipping ((\'zip:/vmfs/volumes/datastore1/utilities/ProFTPD-1/ProFTPD-1.3.3-8-offline_bundle.zip?index.xml\', \'\', "Error extracting index.xml from /vmfs/volumes/datastore1/utilities/ProFTPD-1/ProFTPD-1.3.3-8-offline_bundle.zip: [Errno 2] No such file or directory: \'/vmfs/volumes/datastore1/utilities/ProFTPD-1/ProFTPD-1.3.3-8-offline_bundle.zip\'"))')

    ReplyDelete
    Replies
    1. Hi Dan,

      looks like you are using the wrong path. Is the package really stored in the directory /vmfs/volumes/datastore1/utilities/ProFTPD-1 ?

      Test in a shell with

      ls -l /vmfs/volumes/datastore1/utilities/ProFTPD-1

      Andreas

      Delete
    2. Your are correct! The "u" in "utilities" should have been capital "U". Changed that and it installed. Thanks!

      Delete
  12. Hey Andreas,
    can you post the "missing entry" in /etc/hosts?
    I have the same problem like Gustavo, Alex Kozlov and Sam McHan.
    Thank you <3

    ReplyDelete
    Replies
    1. Just configure a proper hostname in the DNS settings (using the vSphere client or the DCUI). This will automatically add the missing line to /etc/hosts. Something like

      1.2.3.4 your_host_name

      Andreas

      Delete
  13. Hallo Alex,

    der FTP-Server ist sehr hilfreich bei der Übertragung von ca. 3 TB Daten zwischen zwei ESXi-Servern. Er läuft auch unter 5.5, aber dort sorgt er nach Übertragungen für eine Dauer-CPU-Belastung von 33% (bei 2 QC-CPUs). Man muss dann immer den Prozess stoppen und neu starten, was teilweise nur noch per kill auf der CLI möglich ist oder einen Reboot des Hosts erfordert.

    Gruß

    Stefan

    ReplyDelete
  14. Hello Andreas,

    this FTP server is very helpfull for transmitting about 3 TB of data between two ESXi hosts. It's possible to run it under ESXi 5.5 but there are often high CPU loads after data transmissions (33% having 2 QC-CPUs). You then have to stop and restart the proftpd process but often this must be done by the kill command via SSH or by reboot the ESXi host.

    Kind Regards

    Stefan

    ReplyDelete
    Replies
    1. Vielen Dank / Many thanks for your comments!
      I habe not yet tried it in ESXi 5.5 myself and I'm happy to hear that it also works there (somewhat).

      Delete
  15. Hello Andreas,

    The FTP Client gives me a high cpu rate in ESX 5.5 after some time and I have to reboot to stop this. I uses almost 3 cores at 100% and looking at ESXTOP it's ProFTP. Can I do something to change this.

    Cheers,

    Patrick

    ReplyDelete
  16. Hello Andreas,

    I installed ProFTPD in my esxi 5.1 server, it works great but now something is strange is happening. In the main screen of the esxi host, where used to appear the IP adress of the host and other information, now appear a black screen and "ftp" in white letters in one corner. If I press F2 or F11, the menus to go to configuration or to shutdown the servers appear.
    I tried stopping the FTPD service in the Vsphere client and it doesnt work. Then I tried uninstalling ProFTPD from the host by the command "esxcli software vib remove -n ProFTPD" , but I get an error:

    [InstallationError]
    Error in running ['/etc/init.d/proftpd{, 'stop' 'remove']:
    Return code: 1
    Output: Stopping proftpd
    It is no safe to continue. Please reboot the host inmediately to discard the unfinished update.
    Please refer to the log file for more details.

    In this moment is not possible to reboot the server, and i need to fix it right now.
    I would be glad if you can help me.

    Thanks

    ReplyDelete
    Replies
    1. Hi Nicolas,

      I'm sorry that you ran into a problem with the daemon. You can try to stop it in an ESXi shell using "/etc/init.d/proftpd stop" or kill the associated processes.
      The log file to check is /var/log/esxupdate.log, but it just looks like the uninstall routine fails to stop the daemon, because it is somewhat hung.

      You might be able to recover from this only by rebooting the host.

      Andreas

      Delete
    2. Still with the same error, I cant unistall ProFTPD. Is there any other way to unninstall the package?

      Delete
    3. Hi Nicolas,

      can you manually stop the service before uninstalling it?
      If not then set it to "Start and Stop manually" in the vSphere Client, reboot and try to uninstall then.

      - Andreas

      Delete
  17. Finally FTP on ESXi w/o hussle! Thank you very much!

    ReplyDelete
  18. Maybe someday you could look into creating a manual for those people who want to try to integrate some other features into esxi 5... like telnetd etc.

    ReplyDelete
    Replies
    1. I already did this. See my Daemon's VIB series starting here:
      http://www.v-front.de/2012/11/a-daemons-vib-building-software-package.html

      Delete
    2. This comment has been removed by the author.

      Delete
  19. Thanks for this package, FTP server is online. Is there a default login and password to use? Thanks.

    ReplyDelete
    Replies
    1. Hi roterro,
      the package does not create any users, but uses the system builtin users.
      Just log in with your root account.
      Andreas

      Delete
  20. Hi Andreas, I have a problem, I hope you can help. I used the proFTPd package from this site and installed it onto an ESXi 5.1 host. It was all working well until I found that no matter what, if I tried to transfer a large file (100Gb) it would seize up my FTP client (tried two different ones) and so I wanted to try different things in order to fix it.

    One of the things I tried was setting the scoreboard file to /dev/null - which was a bit silly, as proFTPd deletes the file when it starts up. Now I have no /dev/null on my installation, and SSH won't start a new session without it. Can you think of a way I can run these two commands that I think will fix it (mknod /dev/null c 1 3 and chmod 666 /dev/null) on the box as root without having physical access or the ability to log in via SSH?

    Thank you for any help! Michael

    ReplyDelete
  21. Hi,

    firtst of all, I would like to mention that we habe vmware esxi 5.1 installed to an USB flash drive. I tried to install the ProFTPD just like it was instructed, but in the end I got an error like 'restart the ftp server'. When I restart the server, then proftpd isn't there anymore :( Does anybody know a solution to this problem?

    If I try to start the proftpd without restarting vmware, then I got this error:

    Call "HostServiceSystem.Start" for object "serviceSystem" on ESXi "100.140.120.50" failed.

    Thanks for any help in advance!

    ReplyDelete
    Replies
    1. Hi Pablo,

      please check if you have the same issue than what is described in this comment: http://www.v-front.de/2012/11/release-proftpd-ftp-server-for-vmware.html?showComment=1358618460957#c2692394555541127104

      If yes then add your hostname to /etc/hosts and retry.

      Andreas

      Delete
  22. ProFTPD service does not start on ESXi 5.5.0 Update 2.. is there a plan to get an updated version that works?

    ReplyDelete
    Replies
    1. Hi Parag,

      it still does work on ESXi 5.5 U2. You probably have the issue that is described in this other comment.

      Andreas

      Delete
    2. I have this issue too.
      I change "Host Image Profile Acceptance Level" from "Partner Supported" to "Community Supported" via the Vsphere Client.
      It works for me.

      Delete
  23. Hi Andrea,
    thanx for your tutorial, it's wery useful!
    I have a question: in my setup I can download files from the ESXi datastores at a very high datarate (around 100MB/s), while uploading the speed is low (around 5MB/s). Any idea?

    Alberto

    ReplyDelete
    Replies
    1. Hi Alberto82,

      may be caused by your NIC's auto-negotiation settings, or you using an asymmetric line? Try http://bit.ly/1q0Fv6t ...

      Andreas

      Delete
  24. Ahah, thanks for the joke.
    I forgot to specify that I'm not using a remote connection, but a Gigabit lan, full duplex, no autonegotiation in both NICs. And yes, it is simmetric. If that had been the problem, I wouldn't had bothered you for help for sure....

    Thanks,
    Alberto

    ReplyDelete
    Replies
    1. What about the DNS setup? Is there a chance that slow disk I/O is limiting you?

      Is uploading through the vSphere Client's datastore browser also that slow?

      Delete
  25. Thank you so much for ProFTPD. The Daemon's VIB article was very enlightening. Reading VisorFS next !

    ReplyDelete
  26. I have installed ProFTPD on both ESXI 5.5 and 6.0 and it works like a charm. Installation went OK with no issues or errors.

    Andreas, thanks for sharing such a cool tool. It really makes transferring large file easy and faster than using VMware Database Bowser.

    JoeZ

    ReplyDelete
  27. Thank you very much for this! Went from 7MB/s to 88MB/s from SSH to FTP. Saved me many hours of migrating from an old system to another system without the ability to do shared storage. One thing I found was that the security permissions wouldnt let me install from the root folder. I had to put it into a datastore to install for some reason.

    ReplyDelete
  28. Andeas,
    When I try to install your program this is the error I get.
    The ESXi Shell can be disabled by an administrative user. See the
    vSphere Security documentation for more information.
    ~ # cd /vmfs/volumes\datastore1
    -sh: cd: can't cd to /vmfs/volumesdatastore1
    ~ # cd /vmfs/volumes/datastore1
    /vmfs/volumes/54e4a18a-53288094-fb9d-b083fed174e1 # esxcli software vib install
    --no-sig-check -d /vmfs/volumes/datastore1/ProFTPD-1.3.3-8-offline_bundle.zip
    Installation Result
    Message: Host is not changed.
    Reboot Required: false
    VIBs Installed:
    VIBs Removed:
    VIBs Skipped: VFrontDe_bootbank_ProFTPD_1.3.3-8
    /vmfs/volumes/54e4a18a-53288094-fb9d-b083fed174e1 #


    Any help would be greatly appreciated.

    ReplyDelete
    Replies
    1. Hi Scott,

      this message means that the package is already installed.

      Andreas

      Delete
  29. I am glad to tell everyone that I have successfully compiled the last version of proftpd 1.3.6 for ESXi. However, I am not sure if this will solve the high CPU loads. Maybe more testing needs to be done.

    the version enable almost all of the 1.3.6 feature options:
    Autoshadow support,Controls support,curses support,DSO support,IPv6 support,Largefile support,ncurses support,NLS support,Shadow file support,Sendfile support,Trace support,xattr support.
    also include modules:
    mod_core,mod_xfer,mod_rlimit,mod_auth_unix,mod_auth_file,mod_auth,mod_ls,mod_log,mod_site,mod_delay,mod_facts,mod_dso,mod_ident,mod_auth_pam,mod_ban,mod_deflate,mod_quotatab,mod_exec,mod_ratio,mod_readme,mod_wrap2,mod_wrap2_file,mod_wrap2_sql,mod_dnsbl,mod_dynmasq,mod_ifversion,mod_load,mod_log_forensic,mod_radius,mod_rewrite,mod_site_misc,mod_snmp,mod_unique_id,mod_cap,mod_ctrls,mod_lang,mod_ifsession.

    I will release the Proftpd 1.3.6 packages for ESXi tomorrow. coming soon…

    ReplyDelete
  30. Hi, everyone,
    I have packaged ProFTPD 1.3.6 for ESXi and release it now as below.
    It can automatically perform the correct encoding conversion.
    Available for everyone to download. enjoy:)
    You can share these files anywhere or save.

    ProFTPD-1.3.6-offline_bundle.zip
    https://drive.google.com/open?id=1syTHMIfpdK-NJHSQozvhcVpA4Juxkv3Y

    ProFTPD-1.3.6.x86_64.vib
    https://drive.google.com/open?id=1Qk4vHDeWVPSAk4y5C9W5bewL3c4MJd4Y

    After I test it myself, I found that there was still a high CPU load every 2 hours and the service will crashed. you must restart the service.....

    I think this is a long-term bug issues of ProFTPD for processing TCP keepalive.
    I will stop to maintain Proftpd for ESXi version until the bug has been fixed by ProFTPD. forgive me.

    ReplyDelete
    Replies
    1. Anyway, nice job - thanks for sharing!

      Delete
    2. Thank you.
      However, there is still a good news.
      Currently I have finished compiling another FTP service called PureFTP 1.0.47.
      I'm still testing this package.
      So far, the CPU utilization is very low and still has been very stable.
      It can also automatically perform the correct encoding conversion.
      with the following features:
      altlog support, puredb support, throttling support, peruserlimits support, cookie support, diraliases support, extauth support, ftpwho support, pam support, paranoidmsg support, quotas support, ratios support, sysquotas support, uploadscript support, virtualchroot support, virtualhosts support, welcomemsg support, bonjour support, rfc2640 support.

      I think this package should be the best FTP solution for ESXi. After testing I will soon release this package. coming soon…

      Delete
  31. hello, everyone,

    I am glad to release pureftp 1.0.47 for esxi.
    After testing for many days,
    I have found this package is very stable,
    and cpu loading nerver appear to be high in the esxi server.

    In the future, If you have noticed the update of proftpd version,
    you are welcome to inform me. I will synchronize to update the new version pureftp for esxi.

    what is the link to download the package?
    as below, enjoy:

    PureFTPD-1.0.47-offline_bundle.zip
    https://drive.google.com/open?id=1ttIOf7dN3_qvouPhnhlmIUgtOdvs3451

    PureFTPD-1.0.47.x86_64.vib
    https://drive.google.com/open?id=1jTzvSFkwwi84ulJbdUXUYDeJc4Uz4aoU

    ReplyDelete
    Replies
    1. Good work, Chernsha!
      But you have misconfigured PureFTPD (or firewall rules). In order for your package to work, you need either uncomment and change PassivePort parameter in pure-ftpd.conf from 30000 50000 to 49152 49999 (as your firewall rule is), or fix ports range in firewall rule - because it is 30000-50000 by default.

      Delete
  32. thank you, PointLess.

    I have fixed the firewall rules.
    the fixed package as below:

    PureFTPD-1.0.47-offline_bundle.zip
    https://drive.google.com/open?id=1Ok5L3874m64bLqpmqfe0eOt-L7AQ1Drk

    PureFTPD-1.0.47.x86_64.vib
    https://drive.google.com/open?id=1ceYHhLP_JNfl8GQ75EXZcvW4yxP6xppA

    remove previous package first and then install this.
    enjoy.

    ReplyDelete
  33. I forgot to uncomment the PassivePort parameter.
    I have fixed package as below and now the PASV mode work normally.

    PureFTPD-1.0.47-offline_bundle.zip
    https://drive.google.com/open?id=1b6Jkllz8LzPN-d3IT4ZR6M3_TSdrMrZW

    PureFTPD-1.0.47.x86_64.vib
    https://drive.google.com/open?id=1lBZRRqrQ9k2cKo_MFJRoEO8eHjKDRO24

    remove previous package first in maintenance mode and then install this new one.

    ReplyDelete

***** All comments will be moderated! *****
- Please post only comments or questions that are related to this post's contents!
- Advertising and link spamming will not be tolerated!