Troubleshooting: mapping between a SmartCard certificate and an IdM user

Authentication with a SmartCard may fail when the SmartCard certificate is not linked to any IdM user, or to a user different from the one specified on the console.

In order to find which user is associated to a given SmartCard certificate, you can run the following command:

ipaclient$ ipa certmap-match cert.pem
--------------
1 user matched
--------------
 Domain: IPADOMAIN.COM
 User logins: demosc1
----------------------------
Number of entries returned 1
----------------------------

If the result is not what you were expecting, you need first to check how certificates are mapped to users.

By default, a certificate is associated to a user when the user entry contains the full certificate in its usercertificate attribute. But this behavior can be modified by defining certificate mapping rules:

ipaclient$ ipa certmaprule-find
-------------------------------------------
1 Certificate Identity Mapping Rule matched
-------------------------------------------
 Rule name: rulesmartcard
 Mapping rule: (ipacertmapdata=X509:<I>{issuer_dn}<S>{subject_dn})
 Matching rule: <ISSUER>CN=Smart Card CA,O=EXAMPLE.ORG
 Enabled: FALSE
----------------------------
Number of entries returned 1
----------------------------

 

Mapping with full certificate content

When the mapping is based on the full certificate content, you can check if the user entry contains the certificate:

root@ipaclient$ ipa user-show demosc1
 User login: demosc1
[...]
Certificate: MIIC...

If it is not the case, associate the certificate with the user entry using:

ipaclient$ CERT=`cat cert.pem | tail -n +2 | head -n -1 | tr -d '\r\n'`
ipaclient$ ipa user-add-cert demosc1 --certificate $CERT

Once this is done, you may need to clear sssd cache to force SSSD to reload the entries before retrying ipa certmap-match:

ipaclient$ sudo sss_cache -E

 

Flexible mapping with certificate identity mapping rule

When the mapping is based on certificate mapping rules, the same tool ipa certmap-match can be used to check which user entry is associated to a certificate. When the result is not what you expect, you can enable sssd domain logs by adding the following in /etc/sssd/sssd.conf on IdM master:

[domain/ipadomain.com]
...
debug_level = 9

then restart sssd with

root@ipaserver$ systemctl restart sssd

The logs will be located in /var/log/sssd/sssd_ipadomain.log.

 

Check that the certificate identity mapping rules are properly loaded

When sssd is restarted, it reads the mapping rules and should print the following in /var/log/sssd/sssd_ipadomain.log:

[sssd[be[ipadomain.com]]] [sss_certmap_init] (0x0040): sss_certmap initialized.
[sssd[be[ipadomain.com]]] [ipa_certmap_parse_results] (0x4000): Trying to add rule [rulesmartcard][-1][<ISSUER>CN=Smart Card CA,O=EXAMPLE.ORG][(ipacertmapdata=X509:<I>{issuer_dn!nss_x500}<S>{subject_dn!nss_x500})].

If the rule has an invalid syntax, you will see instead:

[sssd[be[ipadomain.com]]] [sss_certmap_init] (0x0040): sss_certmap initialized.
[sssd[be[ipadomain.com]]] [ipa_certmap_parse_results] (0x4000): Trying to add rule [rulesmartcard][-1][<ISSUER>CN=Smart Card CA,O=EXAMPLE.ORG][(ipacertmapdata=X509:<I>{issuer_dn!x509}<S>{subject_dn})].
[sssd[be[ipadomain.com]]] [parse_template] (0x0040): Parse template invalid.
[sssd[be[ipadomain.com]]] [parse_ldap_mapping_rule] (0x0040): Failed to add template.
[sssd[be[ipadomain.com]]] [parse_mapping_rule] (0x0040): Failed to parse LDAP mapping rule.
[sssd[be[ipadomain.com]]] [ipa_certmap_parse_results] (0x0020): sss_certmap_add_rule failed for rule [rulesmartcard], skipping. Please check for typos and if rule syntax is supported.
[sssd[be[ipadomain.com]]] [ipa_subdomains_certmap_done] (0x0040): Unable to parse certmap results [22]: Invalid argument
[sssd[be[ipadomain.com]]] [ipa_subdomains_refresh_certmap_done] (0x0020): Failed to read certificate mapping rules [22]: Invalid argument

The log shows that the rule named rulesmartcard is invalid. Check the rule (see man page for sss-certmap for the supported syntax) and fix if needed:

ipaclient$ ipa certmaprule-show rulesmartcard
 Rule name: rulesmartcard
 Mapping rule: (ipacertmapdata=X509:<I>{issuer_dn!x509}<S>{subject_dn})
 Matching rule: <ISSUER>CN=Smart Card CA,O=EXAMPLE.ORG
 Enabled: TRUE
ipaclient$ ipa certmaprule-mod rulesmartcard --maprule '(ipacertmapdata=X509:<I>{issuer_dn}<S>{subject_dn})'

 

Check that the matching rule corresponds to the certificate

When SSSD tries to associate the certificate to a user, it starts by finding which rule can be applied based on the matching rule (for instance rulesmartcard applies to all certificates issued by CN=Smart Card CA,O=EXAMPLE.ORG because its matching rule is <ISSUER>CN=Smart Card CA,O=EXAMPLE.ORG).

If no matching rule applies to the certificate, SSSD will not be able to associate the certificate with a user, and will display the following in /var/log/sssd/sssd_ipadomain.log:

[sssd[be[ipadomain.com]]] [sss_cert_derb64_to_ldap_filter] (0x0040): Certificate does not match matching-rules.

In this case, you need to create or modify an identity mapping rule, so that the match rule applies to your certificate. See sss-certmap man page for the supported syntax of the –matchrule option of ipa certmaprule-add command.

Check that the expected certificate identity mapping rule is used

When SSSD tries to find the user associated to the certificate, you will see the following logs in /var/log/sssd/sssd_ipadomain.log:

[sssd[be[ipadomain.com]]] [dp_get_account_info_handler] (0x0200): Got request for [0x14][BE_REQ_BY_CERT][cert=MII..]
...
[sssd[be[ipadomain.com]]] [sdap_search_user_next_base] (0x0400): Searching for users with base [cn=accounts,dc=ipadomain,dc=com]
[sssd[be[ipadomain.com]]] [sdap_print_server] (0x2000): Searching 10.34.58.20:389
[sssd[be[ipadomain.com]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(ipacertmapdata=X509:<I>O=EXAMPLE.ORG,CN=Smart Card CA<S>CN=test,O=EXAMPLE.ORG)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=ipadomain,dc=com].
...
[sssd[be[ipadomain.com]]] [sdap_search_user_process] (0x0400): Search for users, returned 0 results.

The logs show the LDAP search performed by SSSD: IP address of the LDAP server, base and search filter. Carefully review this information and compare with what you would expect.

Check that the mapping rule defines a valid search filter

If the rule cannot be transformed to a valid search filter, you will see in /var/log/sssd/sssd_ipadomain.log:

[sssd[be[ipadomain.com]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(ipacertmapdata=X509:<I>O=EXAMPLE.ORG,CN=Smart Card CA<S>CN=test,O=EXAMPLE.ORG(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=ipadomain,dc=com].
[...]
[sssd[be[ipadomain.com]]] [sdap_get_generic_ext_step] (0x0080): ldap_search_ext failed: Bad search filter

If it is the case, you need to fix the certmap rule using

ipaclient$ ipa certmaprule-mod rulesmartcard –maprule …

 

 

Advertisements

Troubleshooting: authentication to the system console or Gnome Desktop Manager of an IdM host with a SmartCard

IdM allows to authenticate to an IdM enrolled-host by providing a SmartCard certificate instead of a username/password. The below steps are based on system console authentication but the process is similar when using Gnome desktop  login authentication.

When the authentication fails, the issue usually comes from a wrong configuration of the IdM system for SmartCard, or of PKINIT.

 

Configuration of the IdM host for SmartCard authentication

If the console does not even prompt for the SmartCard PIN, chances are high that the system was not properly configured for SmartCard authentication.

SSSD configuration for smart card

Check that /etc/sssd.conf contains

[pam]
pam_cert_auth = True

If you need to update the file, do not forget to restart sssd with

root@ipaclient$ systemctl restart sssd

 

SmartCard CA must be trusted

Check that the SmartCard CA is trusted in the /etc/pki/nssdb database:

root@ipaclient$ certutil -L -d /etc/pki/nssdb/

Certificate Nickname Trust Attributes     SSL,S/MIME,JAR/XPI

SmartCardCA                               CT,C,C

If the CA is not present, add it using:

root@ipaclient$ certutil -A -d /etc/pki/nssdb -n SmartCardCA -t CT,C,C -i ca.pem

 

IdM host PKCS#11 module

Check that the IdM host is properly configured for Smart Cards. The opensc package must be installed, the the SmartCard daemon must be running, and the PKCS#11 module must be loaded

root@ipaclient$ dnf install opensc
root@ipaclient$ systemctl start pcscd.service pcscd.socket
root@ipaclient$ modutil -dbdir /etc/pki/nssdb -add "OpenSC" -libfile /usr/lib64/opensc-pkcs11.so

 

Configuration for PKINIT

If the console prompts for the SmartCard PIN but displays

ipaclient login: demosc1
Pin for PIV Card:
Login incorrect

it is possible that the authentication fails trying to acquire a Kerberos ticket with PKINIT. In this case, login with username/password to the IdM host and try to manually perform kinit in order to get more information:

root@ipaclient$ kinit -X X509_user_identity='PKCS11:opensc-pkcs11.so' demosc1

 

If  the command outputs the following:

kinit: Pre-authentication failed: Failed to verify own certificate (depth 1): self signed certificate in certificate chain while getting initial credentials

then check the content of /etc/krb5.conf on the IdM host. The realms section must contain a configuration for ipadomain with pkinit_anchors:

[realms]
 IPADOMAIN.COM = {
 pkinit_anchors = FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem
 pkinit_pool = FILE:/var/lib/ipa-client/pki/ca-bundle.pem

}

The file defined in pkinit_anchors must exist, be readable and contain the certificate of the CA which signed the SmartCard certificate. If it is not the case, run the following commands on any IPA server:

root@ipaserver$ ipa-cacert-manage install -n SmartCardCA -t CT,C,C -p $DM_PWD ca.pem
root@ipaserver$ ipa-certupdate

And run the ipa-certupdate command on all IdM hosts in order to download the certificate.

If the kinit command output the following:

kinit: Certificate mismatch while getting initial credentials

check that the SmartCard certificate is associated to the username provided in the console (see mapping between a SmartCard certificate and an IdM user).

Troubleshooting: ssh to an IdM host with a SmartCard

IdM allows to perform ssh from a non-enrolled host into an IdM enrolled host, using Smart Card authentication instead of ssh authorized keys. The ssh command would be the following to log as demosc1 into the host ipaclient.ipadomain.com:

localuser@localhost$ ssh -I /usr/lib64/opensc-pkcs11.so -l demosc1 ipaclient.ipadomain.com
Enter PIN for 'PIV_II (PIV Card Holder pin)':

The -I option specifies a PKCS#11 shared library, and -l the username on the remote host.

 

Configuration of the local host

First check that the local host is properly configured for Smart Cards. The opensc package must be installed, and the the SmartCard daemon must be running.

localuser@localhost$ sudo dnf install opensc
localuser@localhost$ sudo systemctl start pcscd.service pcscd.socket

 

Configuration of the remote (IdM) host

When IdM is properly configured, ssh will prompt for the SmartCard PIN and authenticate the user. If there is an issue with the certificate, ssh will revert to another authentication type (private keys or username/password).

In this case, enable debug logs for ssh authentication on IdM host. Edit /etc/sssd/sssd.conf and add the following line in [ssh] section:

[ssh]
debug_level = 9

then restart sssd using

root@ipaclient$ systemctl restart sssd

The logs will be located on the IdM host in /var/log/sssd/sssd_ssh.log.

 

The Smart Card CA is not trusted by SSSD

If you see the following in /var/log/sssd/sssd_ssh.log:

[sssd[ssh]] [cert_to_ssh_key] (0x0020): CERT_VerifyCertificateNow failed [-8179].
[sssd[ssh]] [get_valid_certs_keys] (0x0040): cert_to_ssh_key failed, ignoring.

then it means that the CA that signed the Smart Card certificate is not trusted. The trusted certs are stored in /etc/pki/nssdb and can be found using:

root@ipaclient$ certutil -L -d /etc/pki/nssdb

Certificate Nickname Trust Attributes     SSL,S/MIME,JAR/XPI

SmartCardCA                               CT,C,C

 

If the CA cert is missing, add it using

root@ipaclient$ certutil -A -d /etc/pki/nssdb -n SmartCardCA -t CT,C,C -i ca.pem

 

The user is not an IdM user

If the ssh operation does not log any line in /var/log/sssd/sssd_ssh.log, it probably means that the supplied user name is not a user defined in IdM. You can check with:

root@ipaclient$ ipa user-find demosc1
---------------
0 users matched
---------------
----------------------------
Number of entries returned 0
----------------------------

Check that you provided the right user name, or define an IdM user and associate the SmartCard certificate with this user.

 

The certificate is not mapped to the IdM user

If you see the following in /var/log/sssd/sssd_ssh.log:

Found 1 entries in domain ipadomain.com

but the authentication fails, check that the SmartCard certificate is associated to the provided username (refer to mapping between a SmartCard certificate and an IdM user)

Troubleshooting: Authentication to IdM WebGUI with a SmartCard

IdM web GUI can be accessed at the following url: https://ipaserver.ipadomain.com/ipa/ui

The authentication can be done either through Kerberos, by providing a username and password, or with a certificate. If the certificate authentication fails, the issue can either come from your local browser configuration, or from IdM configuration.

The browser does not prompt for the SmartCard PIN

In this case, it is likely that your browser is not properly configured for Smart Card authentication. Please refer to the browser documentation.

 

The browser prompts for the SmartCard PIN but authentication fails

The issue is likely to be a configuration problem on IdM server. The file /var/log/http/error_log will be your best friend to diagnose and fix the issue.

The root CA is not trusted by IdM server

If you see the following in /var/log/http/error_log:

[:error] [pid 50892] Re-negotiation handshake failed: Not accepted by client!?
[:error] [pid 50892] SSL Library Error: -12285 Unable to find the certificate or key necessary for authentication

then it means that the CA that issued your Smart Card certificate is not trusted by Apache httpd server. You need to run the following command on IdM master:

root@ipaserver$ ipa-cacert-manage install -n SmartCardCA -t CT,C,C -p $DM_PWD ca.pem
root@ipaserver$ ipa-certupdate
root@ipaserver$ systemctl restart httpd

The first command adds the Smart Card CA as a trusted CA into IdM. The second command installs the certificate in the various databases used by IdM, especially in /etc/httpd/alias used by Apache. The third command restarts Apache Web server.

The ipa-certupdate step must be run on all IdM hosts, and Apache must be restarted on all IdM servers.

 

Apache is not allowed to authenticate on behalf of a client

If you see the following log in /var/log/http/error_log:

[auth_gssapi:error] [pid 55807] [client 10.40.204.99:60644] GSS ERROR gss_init_sec_context(): [Unspecified GSS failure. Minor code may provide more information (Credential cache is empty)], referer: https://ipaserver.ipadomain.com/ipa/ui/
[:error] [pid 55743] ipa: INFO: 401 Unauthorized: KRB5CCNAME not set

then it means that you forgot to configure apache for delegation. You need to run the following command:

root@ipaserver$ ipa service-mod --ok-to-auth-as-delegate=True HTTP/$(hostname)
root@ipaserver$ systemctl restart httpd

on each IdM server.

 

The SmartCard certificate does not match any IdM user

If you see the following log in /var/log/http/error_log:

[lookup_identity:error] [pid 69382] [client 10.40.204.99:39400] lookup_user_by_certificate failed [dbus_connection_send_with_reply_and_block(org.freedesktop.sssd.infopipe.Users.FindByNameAndCertificate)]: [User not found], referer: https://ipaserver.ipadomain.com/ipa/ui/
[lookup_identity:error] [pid 69382] [client 10.40.204.99:39400] lookup_user_by_certificate cleared r->user, referer: https://ipaserver.ipadomain.com/ipa/ui/
[core:error] [pid 69382] [client 10.40.204.99:39400] AH00027: No authentication done but request not allowed without authentication for /ipa/session/login_x509. Authentication not configured?, referer: https://ipaserver.ipadomin.com/ipa/ui/

then it means that IPA did not find any user associated to the provided certificate. Check  mapping between a SmartCard certificate and an IdM user.

 

The SmartCard certificate matches more than one user

If you see the following log in /var/log/http/error_log:

[lookup_identity:error] [pid 70244] [client 10.40.204.99:39458] lookup_user_by_certificate failed [dbus_connection_send_with_reply_and_block(org.freedesktop.sssd.infopipe.Users.FindByNameAndCertificate)]: [More than one user found. Use ListByCertificate to get all.], referer: https://ipaserver.ipadomain.com/ipa/ui/
[lookup_identity:error] [pid 70244] [client 10.40.204.99:39458] lookup_user_by_certificate cleared r->user, referer: https://ipaserver.ipadomain.com/ipa/ui/
[core:error] [pid 70244] [client 10.40.204.99:39458] AH00027: No authentication done but request not allowed without authentication for /ipa/session/login_x509. Authentication not configured?, referer: https://ipaserver.ipadomain.com/ipa/ui/

then check if the certificate is mapped to a single user using

root@ipaserver$ ipa certmap-match cert.pem
--------------
1 user matched
--------------
 Domain: IPADOMAIN.COM
 User logins: demosc1
----------------------------
Number of entries returned 1
----------------------------

If the certificate matches multiple users, you need to specify the username in the Web GUI.

Apache mod_lookup_identity badly configured

If you see the following in /var/log/http/error_log:

[lookup_identity:error] [pid 70069] [client 10.40.204.99:39444] lookup_user_by_certificate failed [dbus_connection_send_with_reply_and_block(org.freedesktop.sssd.infopipe.Users.FindByNameAndCertificate)]: [More than one user found. Use ListByCertificate to get all.], referer: https://ipaserver.ipadomain/ipa/ui/
[lookup_identity:error] [pid 70069] [client 10.40.204.99:39444] lookup_user_by_certificate cleared r->user, referer: https://ipaserver.ipadomain.com/ipa/ui/
[core:error] [pid 70069] [client 10.40.204.99:39444] AH00027: No authentication done but request not allowed without authentication for /ipa/session/login_x509. Authentication not configured?, referer: https://ipaserver.ipadomain.com/ipa/ui/

then check that /etc/httpd/conf.d/ipa.conf contains the following:

<Location "/ipa/session/login_x509">
...
 LookupUserByCertificate On
 LookupUserByCertificateParamName "username"
...

 

If you see the following in /var/log/http/error_log:

[auth_gssapi:error] [pid 70144] [client 10.40.204.99:39454] GSS ERROR In S4U2Self: gss_acquire_cred_impersonate_name(): [A required input parameter could not be read, No credentials were supplied, or the credentials were unavailable or inaccessible (Unknown error)], referer: https://ipaserver.ipadomain.com/ipa/ui/
[:error] [pid 70141] ipa: INFO: 401 Unauthorized: KRB5CCNAME not set

then check that /etc/httpd/conf.d/ipa.conf contains the following:

<Location "/ipa/session/login_x509">
...
 LookupUserByCertificate On
 LookupUserByCertificateParamName "username"
...

FreeIPA: troubleshooting SmartCard authentication

RHEL 7.4 beta is now available, delivering a new version of IPA which contains the support for Smart Card authentication. This feature allows to use a certificate contained in a SmartCard in order to login to IdM WebUI, to ssh to an IdM-enrolled host, or to login to the console or Gnome Desktop Manager of an IdM-enrolled host.

This feature is really powerful but may also seem difficult to troubleshoot. I will explain where to look for additional information when authentication fails, and how to fix the most common issues.

The information is split into posts specific to each authentication method: