LDAP

Module ldap

This plugin extracts the following:

• People
• Names, emails, titles, and manager information for each person
• List of groups

CLI based Ingestion

Install the Plugin

pip install 'acryl-datahub[ldap]'

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
  type: "ldap"
  config:
    # Coordinates
    ldap_server: ldap://localhost

    # Credentials
    ldap_user: "cn=admin,dc=example,dc=org"
    ldap_password: "admin"

    # Options
    base_dn: "dc=example,dc=org"

sink:
  # sink configs

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options

Field	Required	Type	Description	Default
env		string	The environment that all assets produced by this connector belong to	PROD
platform_instance		string	The instance of the platform that all assets produced by this recipe belong to	None
ldap_server	✅	string	LDAP server URL.	None
ldap_user	✅	string	LDAP user.	None
ldap_password	✅	string	LDAP password.	None
base_dn	✅	string	LDAP DN.	None
filter		string	LDAP extractor filter.	(objectClass=*)
attrs_list		Array of string	Retrieved attributes list	None
custom_props_list		Array of string	A list of custom attributes to extract from the LDAP provider.	None
drop_missing_first_last_name		boolean	If set to true, any users without first and last names will be dropped.	True
page_size		integer	Size of each page to fetch when extracting metadata.	20
user_attrs_map		Dict		{}
group_attrs_map		Dict		{}
stateful_ingestion		StatefulStaleMetadataRemovalConfig (see below for fields)
stateful_ingestion.enabled		boolean	The type of the ingestion state provider registered with datahub.	False
stateful_ingestion.ignore_old_state		boolean	If set to True, ignores the previous checkpoint state.	False
stateful_ingestion.ignore_new_state		boolean	If set to True, ignores the current checkpoint state.	False
stateful_ingestion.remove_stale_metadata		boolean	Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.	True

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "LDAPSourceConfig",
  "description": "Config used by the LDAP Source.",
  "type": "object",
  "properties": {
    "env": {
      "title": "Env",
      "description": "The environment that all assets produced by this connector belong to",
      "default": "PROD",
      "type": "string"
    },
    "platform_instance": {
      "title": "Platform Instance",
      "description": "The instance of the platform that all assets produced by this recipe belong to",
      "type": "string"
    },
    "stateful_ingestion": {
      "$ref": "#/definitions/StatefulStaleMetadataRemovalConfig"
    },
    "ldap_server": {
      "title": "Ldap Server",
      "description": "LDAP server URL.",
      "type": "string"
    },
    "ldap_user": {
      "title": "Ldap User",
      "description": "LDAP user.",
      "type": "string"
    },
    "ldap_password": {
      "title": "Ldap Password",
      "description": "LDAP password.",
      "type": "string"
    },
    "base_dn": {
      "title": "Base Dn",
      "description": "LDAP DN.",
      "type": "string"
    },
    "filter": {
      "title": "Filter",
      "description": "LDAP extractor filter.",
      "default": "(objectClass=*)",
      "type": "string"
    },
    "attrs_list": {
      "title": "Attrs List",
      "description": "Retrieved attributes list",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "custom_props_list": {
      "title": "Custom Props List",
      "description": "A list of custom attributes to extract from the LDAP provider.",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "drop_missing_first_last_name": {
      "title": "Drop Missing First Last Name",
      "description": "If set to true, any users without first and last names will be dropped.",
      "default": true,
      "type": "boolean"
    },
    "page_size": {
      "title": "Page Size",
      "description": "Size of each page to fetch when extracting metadata.",
      "default": 20,
      "type": "integer"
    },
    "user_attrs_map": {
      "title": "User Attrs Map",
      "default": {},
      "type": "object"
    },
    "group_attrs_map": {
      "title": "Group Attrs Map",
      "default": {},
      "type": "object"
    }
  },
  "required": [
    "ldap_server",
    "ldap_user",
    "ldap_password",
    "base_dn"
  ],
  "additionalProperties": false,
  "definitions": {
    "DynamicTypedStateProviderConfig": {
      "title": "DynamicTypedStateProviderConfig",
      "type": "object",
      "properties": {
        "type": {
          "title": "Type",
          "description": "The type of the state provider to use. For DataHub use `datahub`",
          "type": "string"
        },
        "config": {
          "title": "Config",
          "description": "The configuration required for initializing the state provider. Default: The datahub_api config if set at pipeline level. Otherwise, the default DatahubClientConfig. See the defaults (https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/graph/client.py#L19)."
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    },
    "StatefulStaleMetadataRemovalConfig": {
      "title": "StatefulStaleMetadataRemovalConfig",
      "description": "Base specialized config for Stateful Ingestion with stale metadata removal capability.",
      "type": "object",
      "properties": {
        "enabled": {
          "title": "Enabled",
          "description": "The type of the ingestion state provider registered with datahub.",
          "default": false,
          "type": "boolean"
        },
        "ignore_old_state": {
          "title": "Ignore Old State",
          "description": "If set to True, ignores the previous checkpoint state.",
          "default": false,
          "type": "boolean"
        },
        "ignore_new_state": {
          "title": "Ignore New State",
          "description": "If set to True, ignores the current checkpoint state.",
          "default": false,
          "type": "boolean"
        },
        "remove_stale_metadata": {
          "title": "Remove Stale Metadata",
          "description": "Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.",
          "default": true,
          "type": "boolean"
        }
      },
      "additionalProperties": false
    }
  }
}

Code Coordinates

• Class Name: datahub.ingestion.source.ldap.LDAPSource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for LDAP, feel free to ping us on our Slack