My office uses Google Talk for intra-employee instant messaging. This Monday all users got a broadcast message from Google saying that the Google Talk desktop client will cease working on February 15. (Though this may be an old automated notification from when Google was threatening to EOL Talk last February.)

Update (2015-03-09): They finally did kill Talk for Windows as of February 23, 2015.

Of course we can’t take the risk of Google actually shutting down our IMs, and I personally don’t like the new Hangouts Chrome app.

Moreover, we want to limit employees to only messaging other people in our organization. We also don’t necessarily want Google being a party to all of our communication. That means running our own IM server.

ejabberd is a well-known workhorse for IM. Yes, I’ve heard great things about Openfire and others, but I decided to go ejabberd nonetheless.

Environment and Goals

Our organization runs a Windows 2008 AD infrastructure that is very simple: One forest, one domain, one organizational unit. (There’s only about 15 employees.)

I had three goals for our IM solution beyond the obvious ability to chat:

• Authentication to AD
• Interchange of employee particulars (real name, email address, etc.) between the IM server and AD
• Chat contacts defined in AD

The reason for all three of these is that it’s an administrative pain to maintain a separate system and have to create all user accounts twice (something I was doing with Google). Out of the box, ejabberd fit the bill on all three with the following features:

• LDAP authentication
• mod_vcard_ldap
• mod_shared_roster_ldap

We are largely a Windows workstation / Linux server shop, which is why we use Active Directory and I’m running ejabberd on Linux.

It sounded perfect on paper, but in reality it was quite an affair to get it 95% functional.

The Challenges

Installation Woes

By default, the installation binary puts ejabberd in the path /opt/ejabberd-14.12. There are no external dependencies and no config files in etc. Everything is within that directory.

I’m using CentOS, so I followed along with these comments:

The init file doesn’t support chkconfig because it’s missing the required header:

Just adding the following to the beginning of ejabberd.init does the trick:

#!/bin/sh
#
# ejabberd	Startup script for the ejabberd XMPP Server
#
# chkconfig: - 99 10
# description:	ejabberd XMPP server

# Source function library.
. /etc/init.d/functions

set -o errexit

The numbers after chkconfig are boot-up and shutdown priorities, they may need tunning but those seem safe. I don’t think chkconfig is distro-specific, I’ll ask ejabberd developers to add this to the distribution package.

So after adding that to ejabberd.init, all that is left to do is:

cp ejabberd.init /etc/init.d/ejabberd
chmod +x /etc/init.d/ejabberd
/usr/sbin/groupadd -r ejabberd
/usr/sbin/useradd -g ejabberd -p ejabberd -r ejabberd
/sbin/chkconfig --add ejabberd
/sbin/chkconfig ejabberd on

Another issue I ran into was that I installed ejabberd as root, but I was trying to run it as the user ejabberd and so got the error:

sed: can't read /opt/ejabberd-14.12/conf/ejabberd.yml: Permission denied
sed: can't read /opt/ejabberd-14.12/conf/ejabberd.yml: Permission denied
sed: can't read /opt/ejabberd-14.12/conf/ejabberd.yml: Permission denied
sed: can't read /opt/ejabberd-14.12/conf/ejabberd.yml: Permission denied
mkdir: cannot create directory `/opt/ejabberd-14.12/database': Permission denied
./ejabberdctl: line 125: cd: /opt/ejabberd-14.12/database/ejabberd@localhost: Permission denied
sh: /opt/ejabberd-14.12/bin/erl: Permission denied

The solution there is pretty simple: chown -R ejabberd:ejabberd /opt/ejabberd-14.12

Should the ejabberd server fail to start or if it crashes, there may still be ejabberd processes running in the background. That may prevent it from starting again.

# ps -ef |grep ejabberd
ejabberd  1429     1  0 17:04 ?        00:00:00 /opt/ejabberd-14.12/bin/epmd -daemon
ejabberd  1431     1  3 17:04 ?        00:00:04 /opt/ejabberd-14.12/bin/beam.smp -K true -P 250000 -- -root /opt/ejabberd-14.12 -progname /opt/ejabberd-14.12/bin/erl -- -home /opt/ejabberd-14.12 -- -sname ejabberd@localhost -noshell -noinput -noshell -noinput -pa /opt/ejabberd-14.12/lib/ejabberd-14.12/ebin -mnesia dir "/opt/ejabberd-14.12/database/ejabberd@localhost" -ejabberd log_rate_limit 100 log_rotate_size 10485760 log_rotate_count 1 log_rotate_date "" -s ejabberd -sasl sasl_error_logger {file,"/opt/ejabberd-14.12/logs/erlang.log"} -smp auto start

# kill 1429
# kill 1431

Finally, I had one more issue when stopping the service:

service ejabberd stop
Stopping ejabberd...
/opt/ejabberd-14.12/bin/ejabberdctl: line 364: epmd: command not found

Another easy one, because the error message is quite clear. There are a few lines in the file /opt/ejabberd-14.12/bin/ejabberdctl which assume that epmd is in the same directory… which it is! However, ejabberdctl is being invoked by the init script without changing directory to that path.

You can deal with this in three ways:

• Change ejabberdctl to contain the full path of epmd
• Change the init script to cd into /opt/ejabberd-14.12/bin
• Add /opt/ejabberd-14.12/bin to the path

Default SSL Certificate

The first thing I did was start up a plain-Jane configuration of ejabberd and try to connect to it. I was testing it out with both Pidgin and Spark

I don’t remember which of the two clients it was, but one of them refused to connect to the server due to an SSL error. The error message wasn’t explicit, but from what I read online it appeared that an expired certificate would stymie the client. So it seems that the default PEM file included with the ejabberd binaries is invalid.

A self-signed cert isn’t a problem, so I generated one using OpenSSL and gave it an expiration 10 years hence. Not a real issue, but something that you should do right off the bat.

LDAP Authentication

Getting LDAP authentication working wasn’t much of a problem. The documentation is pretty much all you need.

Here’s a sanitized snippet of my LDAP configuration. Note that I created a user with the common name Ejabberd LDAP as an LDAP reader.

auth_method: ldap
ldap_servers:
   - "dcserver01.mydomain.com"
   - "dcserver02.mydomain.com"
ldap_encrypt: none
ldap_port: 389
ldap_rootdn: "CN=Ejabberd LDAP,CN=Users,dc=mydomain,dc=com"
ldap_password: "SomePassword0239"
ldap_base: "cn=Users,dc=mydomain,dc=com"
ldap_uids:
   - "sAMAccountName"
ldap_filter: ""

That’s enough to get LDAP authentication working.

By the way, if you want to get the ldap_rootdn (the user’s Active Directory distinguished name) fire up Active Directory Users and Computers. Open the properties of the user, click on the Attribute Editor tab, and scroll down until you find distinguishedName in the list.

mod_vcard_ldap

This should have been easy, but it was far from it.

First off, at some point the developers of ejabberd switched to a YAML-formatted config file, so most of the examples you’ll find online are in JSON. Once you get a handle on YAML it’s not a big deal to visually bounce between the two, but I’m not accustomed to YAML so there was a bit of a learning curve there. (By the way, “every JSON file is also a valid YAML file“.)

Update (2016-03-25)

Below I point out that a typo in the documentation caused me some grief. I’m leaving that in this post for posterity (or in case you’re referencing an old or archived version of the docs), however Mickaël Rémond from ProcessOne was kind enough to comment that the doc has since been fixed.

Thanks Mickaël, much appreciated!

The real problem was that I copied the mod_vcard_ldap config snippet straight from the docs on Active Directory LDAP integration. Here’s what it says:

  mod_vcard_ldap: 
    ldap_vcard_map: 
      "NICKNAME": {"%u", []}
      "GIVEN": {"%s", ["givenName"]}
      "MIDDLE": {"%s", ["initials"]}
      "FAMILY": {"%s", ["sn"]}
      "FN": {"%s", ["displayName"]}
      "EMAIL": {"%s", ["mail"]}
      "ORGNAME": {"%s", ["company"]}
      "ORGUNIT": {"%s", ["department"]}
      "CTRY": {"%s", ["c"]}
      "LOCALITY": {"%s", ["l"]}
      "STREET": {"%s", ["streetAddress"]}
      "REGION": {"%s", ["st"]}
      "PCODE": {"%s", ["postalCode"]}
      "TITLE": {"%s", ["title"]}
      "URL": {"%s", ["wWWHomePage"]}
      "DESC": {"%s", ["description"]}
      "TEL": {"%s", ["telephoneNumber"]}]}

Do you see the problem? I sure didn’t, and ejabberd was spitting out the following error in the log file:

2015-02-03 17:17:08.699 [error] <0.36.0> CRASH REPORT Process <0.36.0> with 0 neighbours exited with reason: {undefined_macro,''} in application_master:init/4 line 133

Here’s the problem: The documentation’s example uses commas when it should be using colons. That’s it. Of course I didn’t notice/know that for a few hours, so I tried every single bit of nonsense possible to reformat that section into valid YAML.

This is the correct, working mod_vcard_ldap section of my configuration. Note that I pared it down a bit because I don’t populate all of the VCard fields in Active Directory anyhow:

  mod_vcard_ldap:
    ldap_uids: {"sAMAccountName": "%u"}
    ldap_filter: ""
    matches: infinity
    ldap_vcard_map:
      "NICKNAME": {"%s": ["displayName"]}
      "FN": {"%s": ["displayName"]}
      "EMAIL": {"%s": ["mail"]}
      "GIVEN": {"%s": ["givenName"]}
      "MIDDLE": {"%s": ["initials"]}
      "FAMILY": {"%s": ["sn"]}
      "ORGNAME": {"%s": ["company"]}
      "ORGUNIT": {"%s": ["department"]}
      "TITLE": {"%s": ["title"]}
      "TEL": {"%s": ["telephoneNumber"]}
    ldap_search_fields:
      "User": "%u"
      "Full Name":  "displayName"
      "Email": "mail"
    ldap_search_reported:
      "Full Name": "FN"
      "Nickname": "NICKNAME"
      "Email": "EMAIL"

It’s not strictly-speaking necessary to define ldap_uids in this section because ejabberd will use the LDAP settings you’ve previously defined. However you should note that both the documentation and the plurality implied by the field name are inaccurate. In the main LDAP config, the ldap_uids can be a list (array, whatever):

# Correct in main LDAP section
ldap_uids:
   - "sAMAccountName"

In the mod_vcard_ldap section, it is an object/mapping/whatever. An error is thrown if you provide more than one uid field. Also, the second (%u) parameter is required.

# Correct in mod_vcard_ldap section
ldap_uids: {"sAMAccountName": "%u"}

Hopefully my configuration examples here will save you the same headache I faced.

mod_shared_roster

Oh man, was this ever a pain. There are many conflicting examples, many half-baked workarounds, and many compromises to be decided upon.

I’m unclear on whether or not this is still true in the current release, but apparently spaces in a user’s CN causes mod_shared_roster to fail silently.

mikekaganski wrote:
That’s the most trouble, because the “member” stores its members as DNs, thus if you have your users like “CN=John Doe,OU=blah,OU=blah,DC=example,DC=com”, then we’re stuck. In the better case when you have CNs of your users without spaces, you may choose to use ldap_memberattr_format_re = “CN=(\\w*),(OU=.*,)*DC=example,DC=com” (this is from the guide, I didn’t test this regex). Source: https://www.ejabberd.im/node/4826

It took me a while to find that. What does it mean? Well the overly-simplified version of the shared roster LDAP module’s alogrithm goes like this:

1. Run the ldap_rfilter query to get a list of groups that contain Jabber-able contacts
2. For each of those groups, run the ldap_gfilter query to get the group’s displayable name and member list.
3. For each distinct group member retrieved in step 2, get the user’s displayable name.

Everything went wrong for me in step 2. All of my users have their full name as their common name, as you can see below:

# ldapsearch -LLL -H ldap://dcserver01.mydomain.com -x -D 'mydomain\ejabberd.ldap' -w 'SomePassword2098' -E pr=1000/noprompt -b 'dc=mydomain,dc=com' '(&(objectCategory=group)(cn=All Employees))' displayName member

dn: CN=All Employees,CN=Users,DC=mydomain,DC=com
member: CN=Jennifer Doe,CN=Users,DC=mydomain,DC=com
member: CN=Eric Von Lastname,CN=Users,DC=mydomain,DC=com
member: CN=Kieran Wonderbra,CN=Users,DC=mydomain,DC=com
displayName: All Employees

Note that I truncated and sanitized the output above.

As you can see, all the common names have spaces in them, and so it (I believe) is unparsable. I tried variations on the ldap_memberattr_format and ldap_memberattr_format_re properties with no success.

After much hair pulling, this is my working configuration:

  mod_shared_roster_ldap:
    ldap_groupattr: "sAMAccountName"
    ldap_groupdesc: ""
    ldap_memberattr: "sAMAccountName"
    ldap_memberattr_format: "%u"
    ldap_useruid: "sAMAccountName"
    ldap_userdesc: "displayName"
    ldap_rfilter: "(&(objectCategory=group)(cn=All Employees))"
    ldap_gfilter: "(&(objectCategory=user)(memberOf=CN=All Employees,CN=Users,DC=mydomain,DC=com))"
    ldap_ufilter: "(&(objectClass=user)(sAMAccountName=%u))"
    ldap_filter: ""
    ldap_group_cache_validity: 60
    ldap_user_cache_validity: 60
    ldap_auth_check: off

Remember what I said about compromises? Well, my ldap_gfilter query isn’t really giving the module what it wants: A list of group members and the group’s displayable name. That’s why I left ldap_groupdesc blank; My query does not return the group’s name. This means that my group’s name isn’t propagating to the chat clients. The clients simply display a list of contacts outside of any particular group. I have no problem with that, because I only have the one group that contains all employees.

Here’s the output of my ldap_gfilter to contrast with the earlier example that retrieved a group’s members with just their distinguished names:

# ldapsearch -LLL -H ldap://dcserver01.mydomain.com -x -D 'mydomain\ejabberd.ldap' -w 'SomePassword2098' -E pr=1000/noprompt -b 'dc=mydomain,dc=com' 
\ '(&(objectCategory=user)(memberOf=CN=All Employees,CN=Users,DC=mydomain,DC=com))' sAMAccountName

dn: CN=Kieran Wonderbra,CN=Users,DC=mydomain,DC=com
sAMAccountName: kwonderbra

dn: CN=Eric Von Lastname,CN=Users,DC=mydomain,DC=com
sAMAccountName: elastname

dn: CN=Jennifer Doe,CN=Users,DC=mydomain,DC=com
sAMAccountName: jdoe

Again, the output above has been sanitized and truncated for brevity.

I’m now retrieving the sAMAccountName, which I’ve referenced in the ldap_useruid. The contents of sAMAccountName (e.g. jdoe) is then used in the ldap_ufilter in place of %u.

Also, note that I left my group common name (CN) of “All Employees” hard-coded in all three filters. In the ldap_gfilter and the ldap_ufilter fields you can use %g, which will be replaced with the content of the field defined in ldap_groupattr (which is “sAMAccountName” in my example). I only want to have one group which represents all ejabberd users, so this works for me.

Debugging Tools

ldapsearch

Install and run this on your ejabberd server. First off, you’re ensuring that you have connectivity between that server and your LDAP server (e.g. domain controller).

It will also allow you to independently test all of the LDAP filters in your configuration.

ldapsearch is well documented, but here’s an example of its usage as appropriate to my environment:

ldapsearch -LLL -H ldap://dcserver01.mydomain.com -x -D 'mydomain\ejabberd.ldap' -w 'SomePassword2098' -E pr=1000/noprompt -b 'dc=mydomain,dc=com' '(&(objectClass=user)(sAMAccountName=scott))'

That statement will retrieve all of the LDAP attributes for the scott user account.

tcpdump and Wireshark

OK, these are by no means the only appropriate tools out there, but they’re what I used.

In my environment, the traffic between the ejabberd server (in the public server “DMZ” network) and my domain controller (in the private office network) pass through a Linux-based router (i.e. a desktop PC with a lot of NICs). I used tcpdump on the router to capture all of the packets between the two machines on port 389 (LDAP) to a file, and then used scp to transfer the packet dump to my Windows workstation for analysis with Wireshark.

Incidentally, the reason I captured packets at the router instead of on the ejabberd server was so that I could rule out connectivity/firewall issues. The reason I didn’t run Wireshark on my domain controller is because I consider it bad practice to install anything unnecessary on my DC.

Here’s the tcpdump command to get a file that Wireshark will parse (obviously replace the IP address with that of your own ejabberd server):

tcpdump -s 0 -w ~/tcpdump.ldap.20150205-1202.pcap -nnXSvi eth4 "port 389 and host 10.101.1.57"

Also, I chose to use Wireshark rather than reading through the raw tcpdump output because it formats everything quite nicely:

Conclusion

ejabberd does suffer a bit from being long on examples but short on consistency due to its old age. It’s also developed (logically) by Linux/erlang people, and so they’re not as Active Directory LDAP friendly as I’d hoped. (I don’t blame them for that, of course. I went with a Linux-based ejabberd server for a reason.)

On the other hand, ejabberd is quite mature and extremely stable. It’s been a great replacement for the proprietary and non-AD-integrated Google Talk.

I know that I didn’t give a comprehensive how-to guide here, but I’m hoping that at the very least my example configuration snippets point you in the right direction.

As stated in the title, I am using ejabberd 14.12. I installed using the binary installer from the process one website.