As a consequence of the work to support multiple consumer contexts, the syncrepl system now supports full N-Way multimaster replication with entry-level conflict resolution. There are some important constraints, of course: In order to maintain consistent results across all servers, you must maintain tightly synchronized clocks across all participating servers (e.g., you must use NTP on all servers).
<The entryCSNs used for replication now record timestamps with microsecond resolution, instead of just seconds. The delta-syncrepl code has not been updated to support multimaster usage yet, that will come later in the 2.4 cycle.
--- >As a consequence of the work to support multiple consumer contexts, the syncrepl system now supports full N-Way multiprovider replication with entry-level conflict resolution. There are some important constraints, of course: In order to maintain consistent results across all servers, you must maintain tightly synchronized clocks across all participating servers (e.g., you must use NTP on all servers).
>The entryCSNs used for replication now record timestamps with microsecond resolution, instead of just seconds.
=== end appendix-changes.html === === begin appendix-common-errors.html === === end appendix-common-errors.html === === begin appendix-configs.html === === end appendix-configs.html === === begin appendix-contrib.html === === end appendix-contrib.html === === begin appendix-deployments.html === === end appendix-deployments.html === === begin appendix-ldap-result-codes.html === === end appendix-ldap-result-codes.html === === begin appendix-recommended-versions.html === === end appendix-recommended-versions.html === === begin appendix-upgrading.html === === end appendix-upgrading.html === === begin autoconf.html === === end autoconf.html === === begin backends.html === === end backends.html === === begin config.html === 40c40 <slapd(8) includes support for LDAP Sync-based replication, called syncrepl, which may be used to maintain shadow copies of directory information on multiple directory servers. In its most basic configuration, the master is a syncrepl provider and one or more slave (or shadow) are syncrepl consumers. An example master-slave configuration is shown in figure 3.3. Multi-Master configurations are also supported.
--- >slapd(8) includes support for LDAP Sync-based replication, called syncrepl, which may be used to maintain shadow copies of directory information on multiple directory servers. In its most basic configuration, the provider is a syncrepl provider and one or more consumer (or shadow) are syncrepl consumers. An example provider-consumer configuration is shown in figure 3.3. Multi-Provider configurations are also supported.
=== end config.html === === begin copyright.html === === end copyright.html === === begin dbtools.html === === end dbtools.html === === begin glossary.html === === end glossary.html === === begin index.html === 26c26 < 30 January 2020 --- > 11 August 2020 531c531 < 18.2.2. N-Way Multi-Master replication --- > 18.2.2. N-Way Multi-Provider Replication 542c542 < 18.3.3. N-Way Multi-Master --- > 18.3.3. N-Way Multi-Provider 653c653 < A.2.4. N-Way Multimaster Replication --- > A.2.4. N-Way Multiprovider Replication === end index.html === === begin install.html === === end install.html === === begin intro.html === 32c32 <Directories tend to contain descriptive, attribute-based information and support sophisticated filtering capabilities. Directories generally do not support complicated transaction or roll-back schemes found in database management systems designed for handling high-volume complex updates. Directory updates are typically simple all-or-nothing changes, if they are allowed at all. Directories are generally tuned to give quick response to high-volume lookup or search operations. They may have the ability to replicate information widely in order to increase availability and reliability, while reducing response time. When directory information is replicated, temporary inconsistencies between the replicas may be okay, as long as inconsistencies are resolved in a timely manner.
--- >Directories tend to contain descriptive, attribute-based information and support sophisticated filtering capabilities. Directories generally do not support complicated transaction or roll-back schemes found in database management systems designed for handling high-volume complex updates. Directory updates are typically simple all-or-nothing changes, if they are allowed at all. Directories are generally tuned to give quick response to high-volume lookup or search operations. They may have the ability to replicate information widely in order to increase availability and reliability, while reducing response time. When directory information is replicated, temporary inconsistencies between the consumers may be okay, as long as inconsistencies are resolved in a timely manner.
34c34 <A web directory, such as provided by the Open Directory Project <http://dmoz.org>, is a good example of a directory service. These services catalog web pages and are specifically designed to support browsing and searching.
--- >A web directory, such as provided by the Curlie Project <https://curlie.org>, is a good example of a directory service. These services catalog web pages and are specifically designed to support browsing and searching.
129c129,130 <Replication: slapd can be configured to maintain shadow copies of directory information. This single-master/multiple-slave replication scheme is vital in high-volume environments where a single slapd installation just doesn't provide the necessary availability or reliability. For extremely demanding environments where a single point of failure is not acceptable, multi-master replication is also available. slapd includes support for LDAP Sync-based replication.
--- >Replication: slapd can be configured to maintain shadow copies of directory information. This single-provider/multiple-consumer replication scheme is vital in high-volume environments where a single slapd installation just doesn't provide the necessary availability or reliability. For extremely demanding environments where a single point of failure is not acceptable, multi-provider replication is also available. With multi-provider replication two or more nodes can accept write operations allowing for redundancy at the provider level.
>slapd includes support for LDAP Sync-based replication.
=== end intro.html === === begin license.html === === end license.html === === begin limits.html === === end limits.html === === begin maintenance.html === 96c96 <Obviously this doesn't cater for any complicated deployments like MirrorMode or N-Way Multi-Master, but following the above sections and using either commercial support or community support should help. Also check the Troubleshooting section.
--- >Obviously this doesn't cater for any complicated deployments like MirrorMode or N-Way Multi-Provider, but following the above sections and using either commercial support or community support should help. Also check the Troubleshooting section.
=== end maintenance.html === === begin monitoringslapd.html === === end monitoringslapd.html === === begin overlays.html === 60c60 < Note: An accesslog database is unique to a given master. It should never be replicated. --- > Note: An accesslog database is unique to a given provider. It should never be replicated. 209c209 <In order to demonstrate how this overlay works, we shall discuss a typical scenario which might be one master server and three Syncrepl slaves.
--- >In order to demonstrate how this overlay works, we shall discuss a typical scenario which might be one provider server and three Syncrepl replicas.
213c213 < chain-uri "ldap://ldapmaster.example.com" --- > chain-uri "ldap://ldapprovider.example.com" 223c223 < updateref "ldap://ldapmaster.example.com/" --- > updateref "ldap://ldapprovider.example.com/" 225,251c225,251 <The chain-tls statement enables TLS from the slave to the ldap master. The DITs are exactly the same between these machines, therefore whatever user bound to the slave will also exist on the master. If that DN does not have update privileges on the master, nothing will happen.
<You will need to restart the slave after these slapd.conf changes. Then, if you are using loglevel stats (256), you can monitor an ldapmodify on the slave and the master. (If you're using cn=config no restart is required.)
<Now start an ldapmodify on the slave and watch the logs. You should expect something like:
<< Sep 6 09:27:25 slave1 slapd[29274]: conn=11 fd=31 ACCEPT from IP=143.199.102.216:45181 (IP=143.199.102.216:389) < Sep 6 09:27:25 slave1 slapd[29274]: conn=11 op=0 STARTTLS < Sep 6 09:27:25 slave1 slapd[29274]: conn=11 op=0 RESULT oid= err=0 text= < Sep 6 09:27:25 slave1 slapd[29274]: conn=11 fd=31 TLS established tls_ssf=256 ssf=256 < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=1 BIND dn="uid=user1,ou=people,dc=example,dc=com" method=128 < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=1 BIND dn="uid=user1,ou=People,dc=example,dc=com" mech=SIMPLE ssf=0 < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=1 RESULT tag=97 err=0 text= < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=2 MOD dn="uid=user1,ou=People,dc=example,dc=com" < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=2 MOD attr=mail < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=2 RESULT tag=103 err=0 text= < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 op=3 UNBIND < Sep 6 09:27:28 slave1 slapd[29274]: conn=11 fd=31 closed < Sep 6 09:27:28 slave1 slapd[29274]: syncrepl_entry: LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) < Sep 6 09:27:28 slave1 slapd[29274]: syncrepl_entry: be_search (0) < Sep 6 09:27:28 slave1 slapd[29274]: syncrepl_entry: uid=user1,ou=People,dc=example,dc=com < Sep 6 09:27:28 slave1 slapd[29274]: syncrepl_entry: be_modify (0) <<
And on the master you will see this:
<< Sep 6 09:23:57 ldapmaster slapd[2961]: conn=55902 op=3 PROXYAUTHZ dn="uid=user1,ou=people,dc=example,dc=com" < Sep 6 09:23:57 ldapmaster slapd[2961]: conn=55902 op=3 MOD dn="uid=user1,ou=People,dc=example,dc=com" < Sep 6 09:23:57 ldapmaster slapd[2961]: conn=55902 op=3 MOD attr=mail < Sep 6 09:23:57 ldapmaster slapd[2961]: conn=55902 op=3 RESULT tag=103 err=0 text= --- >The chain-tls statement enables TLS from the replica to the ldap provider. The DITs are exactly the same between these machines, therefore whatever user bound to the replica will also exist on the provider. If that DN does not have update privileges on the provider, nothing will happen.
>You will need to restart the replica after these slapd.conf changes. Then, if you are using loglevel stats (256), you can monitor an ldapmodify on the replica and the provider. (If you're using cn=config no restart is required.)
>Now start an ldapmodify on the replica and watch the logs. You should expect something like:
>> Sep 6 09:27:25 replica1 slapd[29274]: conn=11 fd=31 ACCEPT from IP=143.199.102.216:45181 (IP=143.199.102.216:389) > Sep 6 09:27:25 replica1 slapd[29274]: conn=11 op=0 STARTTLS > Sep 6 09:27:25 replica1 slapd[29274]: conn=11 op=0 RESULT oid= err=0 text= > Sep 6 09:27:25 replica1 slapd[29274]: conn=11 fd=31 TLS established tls_ssf=256 ssf=256 > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=1 BIND dn="uid=user1,ou=people,dc=example,dc=com" method=128 > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=1 BIND dn="uid=user1,ou=People,dc=example,dc=com" mech=SIMPLE ssf=0 > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=1 RESULT tag=97 err=0 text= > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=2 MOD dn="uid=user1,ou=People,dc=example,dc=com" > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=2 MOD attr=mail > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=2 RESULT tag=103 err=0 text= > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 op=3 UNBIND > Sep 6 09:27:28 replica1 slapd[29274]: conn=11 fd=31 closed > Sep 6 09:27:28 replica1 slapd[29274]: syncrepl_entry: LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) > Sep 6 09:27:28 replica1 slapd[29274]: syncrepl_entry: be_search (0) > Sep 6 09:27:28 replica1 slapd[29274]: syncrepl_entry: uid=user1,ou=People,dc=example,dc=com > Sep 6 09:27:28 replica1 slapd[29274]: syncrepl_entry: be_modify (0) >>And on the provider you will see this:
>> Sep 6 09:23:57 ldapprovider slapd[2961]: conn=55902 op=3 PROXYAUTHZ dn="uid=user1,ou=people,dc=example,dc=com" > Sep 6 09:23:57 ldapprovider slapd[2961]: conn=55902 op=3 MOD dn="uid=user1,ou=People,dc=example,dc=com" > Sep 6 09:23:57 ldapprovider slapd[2961]: conn=55902 op=3 MOD attr=mail > Sep 6 09:23:57 ldapprovider slapd[2961]: conn=55902 op=3 RESULT tag=103 err=0 text= 254c254 < Note: You can clearly see the PROXYAUTHZ line on the master, indicating the proper identity assertion for the update on the master. Also note the slave immediately receiving the Syncrepl update from the master. --- > Note: You can clearly see the PROXYAUTHZ line on the provider, indicating the proper identity assertion for the update on the provider. Also note the replica immediately receiving the Syncrepl update from the provider. 486c486 <--- >
LDAP servers typically hold one or more subtrees of aDIT . Replica (or shadow) servers hold shadow copies of entries held by one or more master servers. Changes are propagated from the master server to replica (slave) servers using LDAP Sync replication. An LDAP cache is a special type of replica which holds entries corresponding to search filters instead of subtrees.=== end overlays.html === === begin preface.html === === end preface.html === === begin quickstart.html === === end quickstart.html === === begin referrals.html === === end referrals.html === === begin replication.html === 26,27c26,27 <
LDAP servers typically hold one or more subtrees of aDIT . Replica (or shadow) servers hold shadow copies of entries held by one or more provider servers. Changes are propagated from the provider server to replica servers using LDAP Sync replication. An LDAP cache is a special type of replica which holds entries corresponding to search filters instead of subtrees.OpenLDAP has various configuration options for creating a replicated directory. In previous releases, replication was discussed in terms of a master server and some number of slave servers. A master accepted directory updates from other clients, and a slave only accepted updates from a (single) master. The replication structure was rigidly defined and any particular database could only fulfill a single role, either master or slave.
<As OpenLDAP now supports a wide variety of replication topologies, these terms have been deprecated in favor of provider and consumer: A provider replicates directory updates to consumers; consumers receive replication updates from providers. Unlike the rigidly defined master/slave relationships, provider/consumer roles are quite fluid: replication updates received in a consumer can be further propagated by that consumer to other servers, so a consumer can also act simultaneously as a provider. Also, a consumer need not be an actual LDAP server; it may be just an LDAP client.
--- >OpenLDAP has various configuration options for creating a replicated directory. In previous releases, replication was discussed in terms of a master server and some number of slave servers. A master accepted directory updates from other clients, and a slave only accepted updates from a (single) master. The replication structure was rigidly defined and any particular database could only fulfill a single role, either master or slave. Another historic term introduced with OpenLDAP 2.4 was multimaster.
>As OpenLDAP now supports a wide variety of replication topologies, these terms have been deprecated in favor of provider/multi-provider and consumer: A provider can accept external write operations and make them available for retrieval by consumers; consumers request replication updates from providers. Unlike the rigidly defined master/slave relationships, provider/consumer roles are quite fluid: replication updates received in a consumer can be further propagated by that consumer to other servers, so a consumer can also act simultaneously as a provider. Also, a consumer need not be an actual LDAP server; it may be just an LDAP client.
32,34c32,34 <The
<LDAP Sync Replication engine,syncrepl for short, is a consumer-side replication engine that enables the consumerLDAP server to maintain a shadow copy of aDIT fragment. A syncrepl engine resides at the consumer and executes as one of the slapd(8) threads. It creates and maintains a consumer replica by connecting to the replication provider to perform the initial DIT content load followed either by periodic content polling or by timely updates upon content changes.Syncrepl uses the LDAP Content Synchronization protocol (or LDAP Sync for short) as the replica synchronization protocol. LDAP Sync provides a stateful replication which supports both pull-based and push-based synchronization and does not mandate the use of a history store. In pull-based replication the consumer periodically polls the provider for updates. In push-based replication the consumer listens for updates that are sent by the provider in realtime. Since the protocol does not require a history store, the provider does not need to maintain any log of updates it has received (Note that the syncrepl engine is extensible and additional replication protocols may be supported in the future.).
<Syncrepl keeps track of the status of the replication content by maintaining and exchanging synchronization cookies. Because the syncrepl consumer and provider maintain their content status, the consumer can poll the provider content to perform incremental synchronization by asking for the entries required to make the consumer replica up-to-date with the provider content. Syncrepl also enables convenient management of replicas by maintaining replica status. The consumer replica can be constructed from a consumer-side or a provider-side backup at any synchronization status. Syncrepl can automatically resynchronize the consumer replica up-to-date with the current provider content.
--- >The
>LDAP Sync Replication engine,syncrepl for short, is a consumer-side replication engine that enables the consumerLDAP server to maintain a shadow copy of aDIT fragment. A syncrepl engine resides at the consumer and executes as one of the slapd(8) threads. It creates and maintains a replica by connecting to the replication provider to perform the initial DIT content load followed either by periodic content polling or by timely updates upon content changes.Syncrepl uses the LDAP Content Synchronization protocol (or LDAP Sync for short) as the consumer synchronization protocol. LDAP Sync provides a stateful replication which supports both pull-based and push-based synchronization and does not mandate the use of a history store. In pull-based replication the consumer periodically polls the provider for updates. In push-based replication the consumer listens for updates that are sent by the provider in realtime. Since the protocol does not require a history store, the provider does not need to maintain any log of updates it has received (Note that the syncrepl engine is extensible and additional replication protocols may be supported in the future.).
>Syncrepl keeps track of the status of the replication content by maintaining and exchanging synchronization cookies. Because the syncrepl consumer and provider maintain their content status, the consumer can poll the provider content to perform incremental synchronization by asking for the entries required to make the consumer up-to-date with the provider content. Syncrepl also enables convenient management of consumers by maintaining replication status. The consumer database can be constructed from a consumer-side or a provider-side backup at any synchronization status. Syncrepl can automatically resynchronize the consumer database to be up-to-date with the current provider content.
36,37c36,37 <With syncrepl, a consumer server can create a replica without changing the provider's configurations and without restarting the provider server, if the consumer server has appropriate access privileges for the DIT fragment to be replicated. The consumer server can stop the replication also without the need for provider-side changes and restart.
<Syncrepl supports partial, sparse, and fractional replications. The shadow DIT fragment is defined by a general search criteria consisting of base, scope, filter, and attribute list. The replica content is also subject to the access privileges of the bind identity of the syncrepl replication connection.
--- >With syncrepl, a consumer can create a replication agreement without changing the provider's configurations and without restarting the provider server, if the consumer server has appropriate access privileges for the DIT fragment to be replicated. The consumer server can stop the replication also without the need for provider-side changes and restart.
>Syncrepl supports partial, sparse, and fractional replications. The shadow DIT fragment is defined by a general search criteria consisting of base, scope, filter, and attribute list. The consumer content is also subject to the access privileges of the bind identity of the syncrepl replication connection.
50c50 <The syncrepl engine utilizes both the present phase and the delete phase of the refresh synchronization. It is possible to configure a session log in the provider which stores the entryUUIDs of a finite number of entries deleted from a database. Multiple replicas share the same session log. The syncrepl engine uses the delete phase if the session log is present and the state of the consumer server is recent enough that no session log entries are truncated after the last synchronization of the client. The syncrepl engine uses the present phase if no session log is configured for the replication content or if the consumer replica is too outdated to be covered by the session log. The current design of the session log store is memory based, so the information contained in the session log is not persistent over multiple provider invocations. It is not currently supported to access the session log store by using LDAP operations. It is also not currently supported to impose access control to the session log.
--- >The syncrepl engine utilizes both the present phase and the delete phase of the refresh synchronization. It is possible to configure a session log in the provider which stores the entryUUIDs of a finite number of entries deleted from a database. Multiple consumers share the same session log. The syncrepl engine uses the delete phase if the session log is present and the state of the consumer server is recent enough that no session log entries are truncated after the last synchronization of the client. The syncrepl engine uses the present phase if no session log is configured for the replication content or if the consumer is too outdated to be covered by the session log. The current design of the session log store is memory based, so the information contained in the session log is not persistent over multiple provider invocations. It is not currently supported to access the session log store by using LDAP operations. It is also not currently supported to impose access control to the session log.
57,58c57,58 <The consumer also stores its replica state, which is the provider's contextCSN received as a synchronization cookie, in the contextCSN attribute of the suffix entry. The replica state maintained by a consumer server is used as the synchronization state indicator when it performs subsequent incremental synchronization with the provider server. It is also used as a provider-side synchronization state indicator when it functions as a secondary provider server in a cascading replication configuration. Since the consumer and provider state information are maintained in the same location within their respective databases, any consumer can be promoted to a provider (and vice versa) without any special actions.
<Because a general search filter can be used in the syncrepl specification, some entries in the context may be omitted from the synchronization content. The syncrepl engine creates a glue entry to fill in the holes in the replica context if any part of the replica content is subordinate to the holes. The glue entries will not be returned in the search result unless ManageDsaIT control is provided.
--- >The consumer also stores its replication state, which is the provider's contextCSN received as a synchronization cookie, in the contextCSN attribute of the suffix entry. The replication state maintained by a consumer server is used as the synchronization state indicator when it performs subsequent incremental synchronization with the provider server. It is also used as a provider-side synchronization state indicator when it functions as a secondary provider server in a cascading replication configuration. Since the consumer and provider state information are maintained in the same location within their respective databases, any consumer can be promoted to a provider (and vice versa) without any special actions.
>Because a general search filter can be used in the syncrepl specification, some entries in the context may be omitted from the synchronization content. The syncrepl engine creates a glue entry to fill in the holes in the consumer context if any part of the consumer content is subordinate to the holes. The glue entries will not be returned in the search result unless ManageDsaIT control is provided.
68c68 <For example, suppose you have a database consisting of 102,400 objects of 1 KB each. Further, suppose you routinely run a batch job to change the value of a single two-byte attribute value that appears in each of the 102,400 objects on the master. Not counting LDAP and TCP/IP protocol overhead, each time you run this job each consumer will transfer and process 100 MB of data to process 200KB of changes!
--- >For example, suppose you have a database consisting of 102,400 objects of 1 KB each. Further, suppose you routinely run a batch job to change the value of a single two-byte attribute value that appears in each of the 102,400 objects on the provider. Not counting LDAP and TCP/IP protocol overhead, each time you run this job each consumer will transfer and process 100 MB of data to process 200KB of changes!
72c72 <Delta-syncrepl, a changelog-based variant of syncrepl, is designed to address situations like the one described above. Delta-syncrepl works by maintaining a changelog of a selectable depth in a separate database on the provider. The replication consumer checks the changelog for the changes it needs and, as long as the changelog contains the needed changes, the consumer fetches the changes from the changelog and applies them to its database. If, however, a replica is too far out of sync (or completely empty), conventional syncrepl is used to bring it up to date and replication then switches back to the delta-syncrepl mode.
--- >Delta-syncrepl, a changelog-based variant of syncrepl, is designed to address situations like the one described above. Delta-syncrepl works by maintaining a changelog of a selectable depth in a separate database on the provider. The replication consumer checks the changelog for the changes it needs and, as long as the changelog contains the needed changes, the consumer fetches the changes from the changelog and applies them to its database. If, however, a consumer is too far out of sync (or completely empty), conventional syncrepl is used to bring it up to date and replication then switches back to the delta-syncrepl mode.
77,79c77,79 <18.2.2. N-Way Multi-Master replication
<Multi-Master replication is a replication technique using Syncrepl to replicate data to multiple provider ("Master") Directory servers.
<18.2.2.1. Valid Arguments for Multi-Master replication
--- >18.2.2. N-Way Multi-Provider Replication
>Multi-Provider replication is a replication technique using Syncrepl to replicate data to multiple provider ("Provider") Directory servers.
>18.2.2.1. Valid Arguments for Multi-Provider replication
85,86c85,86 <18.2.2.2. Invalid Arguments for Multi-Master replication
<(These are often claimed to be advantages of Multi-Master replication but those claims are false):
--- >18.2.2.2. Invalid Arguments for Multi-Provider replication
>(These are often claimed to be advantages of Multi-Provider replication but those claims are false):
89,91c89,91 <
For configuration, please see the N-Way Multi-Master section below
--- >For configuration, please see the N-Way Multi-Provider section below
100c100 <MirrorMode is a hybrid configuration that provides all of the consistency guarantees of single-master replication, while also providing the high availability of multi-master. In MirrorMode two providers are set up to replicate from each other (as a multi-master configuration), but an external frontend is employed to direct all writes to only one of the two servers. The second provider will only be used for writes if the first provider crashes, at which point the frontend will switch to directing all writes to the second provider. When a crashed provider is repaired and restarted it will automatically catch up to any changes on the running provider and resync.
--- >MirrorMode is a hybrid configuration that provides all of the consistency guarantees of single-provider replication, while also providing the high availability of multi-provider. In MirrorMode two providers are set up to replicate from each other (as a multi-provider configuration), but an external frontend is employed to direct all writes to only one of the two servers. The second provider will only be used for writes if the first provider crashes, at which point the frontend will switch to directing all writes to the second provider. When a crashed provider is repaired and restarted it will automatically catch up to any changes on the running provider and resync.
109c109 <The slurpd daemon was the original replication mechanism inherited from UMich's LDAP and operated in push mode: the master pushed changes to the slaves. It was replaced for many reasons, in brief:
--- >The slurpd daemon was the original replication mechanism inherited from UMich's LDAP and operated in push mode: the provider pushed changes to the replicas. It was replaced for many reasons, in brief:
124,125c124,125 <Because syncrepl is a consumer-side replication engine, the syncrepl specification is defined in slapd.conf(5) of the consumer server, not in the provider server's configuration file. The initial loading of the replica content can be performed either by starting the syncrepl engine with no synchronization cookie or by populating the consumer replica by loading an
When loading from a backup, it is not required to perform the initial loading from the up-to-date backup of the provider content. The syncrepl engine will automatically synchronize the initial consumer replica to the current provider content. As a result, it is not required to stop the provider server in order to avoid the replica inconsistency caused by the updates to the provider content during the content backup and loading process.
<When replicating a large scale directory, especially in a bandwidth constrained environment, it is advised to load the consumer replica from a backup instead of performing a full initial load using syncrepl.
--- >Because syncrepl is a consumer-side replication engine, the syncrepl specification is defined in slapd.conf(5) of the consumer server, not in the provider server's configuration file. The initial loading of the consumer content can be performed either by starting the syncrepl engine with no synchronization cookie or by populating the consumer by loading an
When loading from a backup, it is not required to perform the initial loading from the up-to-date backup of the provider content. The syncrepl engine will automatically synchronize the initial consumer to the current provider content. As a result, it is not required to stop the provider server in order to avoid the replication inconsistency caused by the updates to the provider content during the content backup and loading process.
>When replicating a large scale directory, especially in a bandwidth constrained environment, it is advised to load the consumer from a backup instead of performing a full initial load using syncrepl.
184c184 <The syncrepl replication is specified in the database section of slapd.conf(5) for the replica context. The syncrepl engine is backend independent and the directive can be defined with any database type.
--- >The syncrepl directive is specified in the database section of slapd.conf(5) for the consumer context. The syncrepl engine is backend independent and the directive can be defined with any database type.
211c211 <When starting a consumer slapd(8), it is possible to provide a synchronization cookie as the -c cookie command line option in order to start the synchronization from a specific state. The cookie is a comma separated list of name=value pairs. Currently supported syncrepl cookie fields are csn=<csn> and rid=<rid>. <csn> represents the current synchronization state of the consumer replica. <rid> identifies a consumer replica locally within the consumer server. It is used to relate the cookie to the syncrepl definition in slapd.conf(5) which has the matching replica identifier. The <rid> must have no more than 3 decimal digits. The command line cookie overrides the synchronization cookie stored in the consumer replica database.
--- >When starting a consumer slapd(8), it is possible to provide a synchronization cookie as the -c cookie command line option in order to start the synchronization from a specific state. The cookie is a comma separated list of name=value pairs. Currently supported syncrepl cookie fields are csn=<csn> and rid=<rid>. <csn> represents the current synchronization state of the consumer. <rid> identifies a consumer locally within the consumer server. It is used to relate the cookie to the syncrepl definition in slapd.conf(5) which has the matching <rid>. The <rid> must have no more than 3 decimal digits. The command line cookie overrides the synchronization cookie stored in the consumer database.
214c214 <Setting up delta-syncrepl requires configuration changes on both the master and replica servers:
--- >Setting up delta-syncrepl requires configuration changes on both the provider and replica servers:
216c216 < # Give the replica DN unlimited read access. This ACL needs to be --- > # Give the replicator DN unlimited read access. This ACL needs to be 249c249 < # Let the replica DN have limitless searches --- > # Let the replicator DN have limitless searches 276c276 < # Let the replica DN have limitless searches --- > # Let the replicator DN have limitless searches 296c296 < provider=ldap://ldapmaster.example.com:389 --- > provider=ldap://ldapprovider.example.com:389 308,309c308,309 < # Refer updates to the master < updateref ldap://ldapmaster.example.com --- > # Refer updates to the provider > updateref ldap://ldapprovider.example.com 311c311 <The above configuration assumes that you have a replicator identity defined in your database that can be used to bind to the provider. In addition, all of the databases (primary, replica, and the accesslog storage database) should also have properly tuned DB_CONFIG files that meet your needs.
--- >The above configuration assumes that you have a replicator identity defined in your database that can be used to bind to the provider. In addition, all of the databases (primary, replica, and the accesslog storage database) should also have properly tuned DB_CONFIG files that meet your needs if using the bdb or hdb backends.
313c313 < Note: An accesslog database is unique to a given master. It should never be replicated. --- > Note: An accesslog database is unique to a given provider. It should never be replicated. 315,316c315,316 <For the following example we will be using 3 Master nodes. Keeping in line with test050-syncrepl-multimaster of the OpenLDAP test suite, we will be configuring slapd(8) via cn=config
--- >For the following example we will be using 3 Provider nodes. Keeping in line with test050-syncrepl-multiprovider of the OpenLDAP test suite, we will be configuring slapd(8) via cn=config
341c341 <This sets up syncrepl as a provider (since these are all masters):
--- >This sets up syncrepl as a provider (since these are all providers):
349c349 <Now we setup the first Master Node (replace $URI1, $URI2 and $URI3 etc. with your actual ldap urls):
--- >Now we setup the first Provider Node (replace $URI1, $URI2 and $URI3 etc. with your actual ldap urls):
380,381c380,381 <Now start up the Master and a consumer/s, also add the above LDIF to the first consumer, second consumer etc. It will then replicate cn=config. You now have N-Way Multimaster on the config database.
<We still have to replicate the actual data, not just the config, so add to the master (all active and configured consumers/masters will pull down this config, as they are all syncing). Also, replace all ${} variables with whatever is applicable to your setup:
--- >Now start up the provider and a consumer/s, also add the above LDIF to the first consumer, second consumer etc. It will then replicate cn=config. You now have N-Way Multi-Provider on the config database.
>We still have to replicate the actual data, not just the config, so add to the provider (all active and configured consumers/providers will pull down this config, as they are all syncing). Also, replace all ${} variables with whatever is applicable to your setup:
474c474 <You will now have a directory architecture that provides all of the consistency guarantees of single-master replication, while also providing the high availability of multi-master replication.
--- >You will now have a directory architecture that provides all of the consistency guarantees of single-provider replication, while also providing the high availability of multi-provider replication.
481c481 < # Standard OpenLDAP Master/Provider --- > # Standard OpenLDAP Provider 524c524 < # Let the replica DN have limitless searches --- > # Let the replicator DN have limitless searches 566c566 < # Standard OpenLDAP Slave without Syncrepl --- > # Standard OpenLDAP Replica without Syncrepl 589c589 < directory /usr/local/var/openldap-slave/data --- > directory /usr/local/var/openldap-consumer/data 601c601 < # Let the replica DN have limitless searches --- > # Let the replicator DN have limitless searches 606c606 < # Refer updates to the master --- > # Refer updates to the provider 616c616 < # Give the replica DN unlimited read access. This ACL may need to be --- > # Give the replicator DN unlimited read access. This ACL may need to be 640c640 < Note: You must populate the Master and Slave directories with the same data, unlike when using normal Syncrepl --- > Note: You must populate the Provider and Replica directories with the same data, unlike when using normal Syncrepl 642c642 <If you do not have access to modify the master directory configuration you can configure a standalone ldap proxy, which might look like:
--- >If you do not have access to modify the provider directory configuration you can configure a standalone ldap proxy, which might look like:
=== end replication.html === === begin runningslapd.html === === end runningslapd.html === === begin sasl.html === === end sasl.html === === begin schema.html === === end schema.html === === begin security.html === === end security.html === === begin slapdconf2.html === 620c620 <This directive specifies the current database as a replica of the master content by establishing the current slapd(8) as a replication consumer site running a syncrepl replication engine. The master database is located at the replication provider site specified by the provider parameter. The replica database is kept up-to-date with the master content using the LDAP Content Synchronization protocol. See RFC4533 for more information on the protocol.
--- >This directive specifies the current database as a consumer of the provider content by establishing the current slapd(8) as a replication consumer site running a syncrepl replication engine. The provider database is located at the provider site specified by the provider parameter. The consumer database is kept up-to-date with the provider content using the LDAP Content Synchronization protocol. See RFC4533 for more information on the protocol.
622,624c622,624 <The provider parameter specifies the replication provider site containing the master content as an LDAP URI. The provider parameter specifies a scheme, a host and optionally a port where the provider slapd instance can be found. Either a domain name or IP address may be used for <hostname>. Examples are ldap://provider.example.com:389 or ldaps://192.168.1.1:636. If <port> is not given, the standard LDAP port number (389 or 636) is used. Note that the syncrepl uses a consumer-initiated protocol, and hence its specification is located at the consumer site, whereas the replica specification is located at the provider site. syncrepl and replica directives define two independent replication mechanisms. They do not represent the replication peers of each other.
<The content of the syncrepl replica is defined using a search specification as its result set. The consumer slapd will send search requests to the provider slapd according to the search specification. The search specification includes searchbase, scope, filter, attrs, attrsonly, sizelimit, and timelimit parameters as in the normal search specification. The searchbase parameter has no default value and must always be specified. The scope defaults to sub, the filter defaults to (objectclass=*), attrs defaults to "*,+" to replicate all user and operational attributes, and attrsonly is unset by default. Both sizelimit and timelimit default to "unlimited", and only positive integers or "unlimited" may be specified.
<The
The provider parameter specifies the replication provider site containing the provider content as an LDAP URI. The provider parameter specifies a scheme, a host and optionally a port where the provider slapd instance can be found. Either a domain name or IP address may be used for <hostname>. Examples are ldap://provider.example.com:389 or ldaps://192.168.1.1:636. If <port> is not given, the standard LDAP port number (389 or 636) is used. Note that the syncrepl uses a consumer-initiated protocol, and hence its specification is located on the consumer.
>The content of the syncrepl consumer is defined using a search specification as its result set. The consumer slapd will send search requests to the provider slapd according to the search specification. The search specification includes searchbase, scope, filter, attrs, attrsonly, sizelimit, and timelimit parameters as in the normal search specification. The searchbase parameter has no default value and must always be specified. The scope defaults to sub, the filter defaults to (objectclass=*), attrs defaults to "*,+" to replicate all user and operational attributes, and attrsonly is unset by default. Both sizelimit and timelimit default to "unlimited", and only positive integers or "unlimited" may be specified.
>The
The schema checking can be enforced at the LDAP Sync consumer site by turning on the schemachecking parameter. If it is turned on, every replicated entry will be checked for its schema as the entry is stored into the replica content. Every entry in the replica should contain those attributes required by the schema definition. If it is turned off, entries will be stored without checking schema conformance. The default is off.
<The binddn parameter gives the DN to bind as for the syncrepl searches to the provider slapd. It should be a DN which has read access to the replication content in the master database.
--- >The schema checking can be enforced at the LDAP Sync consumer site by turning on the schemachecking parameter. If it is turned on, every replicated entry will be checked for its schema as the entry is stored on the consumer. Every entry in the consumer should contain those attributes required by the schema definition. If it is turned off, entries will be stored without checking schema conformance. The default is off.
>The binddn parameter gives the DN to bind as for the syncrepl searches to the provider slapd. It should be a DN which has read access to the replication content in the provider database.
644c644 <This directive is only applicable in a slave slapd. It specifies the URL to return to clients which submit update requests upon the replica. If specified multiple times, each
This directive is only applicable in a replica (or shadow) slapd(8) instance. It specifies the URL to return to clients which submit update requests upon the replica. If specified multiple times, each
This directive specifies the current database as a replica of the master content by establishing the current slapd(8) as a replication consumer site running a syncrepl replication engine. The master database is located at the replication provider site specified by the provider parameter. The replica database is kept up-to-date with the master content using the LDAP Content Synchronization protocol. See RFC4533 for more information on the protocol.
--- >This directive specifies the current database as a consumer of the provider content by establishing the current slapd(8) as a replication consumer site running a syncrepl replication engine. The provider database is located at the replication provider site specified by the provider parameter. The consumer database is kept up-to-date with the provider content using the LDAP Content Synchronization protocol. See RFC4533 for more information on the protocol.
522,524c522,524 <The provider parameter specifies the replication provider site containing the master content as an LDAP URI. The provider parameter specifies a scheme, a host and optionally a port where the provider slapd instance can be found. Either a domain name or IP address may be used for <hostname>. Examples are ldap://provider.example.com:389 or ldaps://192.168.1.1:636. If <port> is not given, the standard LDAP port number (389 or 636) is used. Note that the syncrepl uses a consumer-initiated protocol, and hence its specification is located at the consumer site, whereas the replica specification is located at the provider site. syncrepl and replica directives define two independent replication mechanisms. They do not represent the replication peers of each other.
<The content of the syncrepl replica is defined using a search specification as its result set. The consumer slapd will send search requests to the provider slapd according to the search specification. The search specification includes searchbase, scope, filter, attrs, exattrs, attrsonly, sizelimit, and timelimit parameters as in the normal search specification. The searchbase parameter has no default value and must always be specified. The scope defaults to sub, the filter defaults to (objectclass=*), attrs defaults to "*,+" to replicate all user and operational attributes, and attrsonly is unset by default. Both sizelimit and timelimit default to "unlimited", and only positive integers or "unlimited" may be specified. The exattrs option may also be used to specify attributes that should be omitted from incoming entries.
<The
The provider parameter specifies the replication provider site containing the provider content as an LDAP URI. The provider parameter specifies a scheme, a host and optionally a port where the provider slapd instance can be found. Either a domain name or IP address may be used for <hostname>. Examples are ldap://provider.example.com:389 or ldaps://192.168.1.1:636. If <port> is not given, the standard LDAP port number (389 or 636) is used. Note that the syncrepl uses a consumer-initiated protocol, and hence its specification is located on the consumer.
>The content of the syncrepl consumer is defined using a search specification as its result set. The consumer slapd will send search requests to the provider slapd according to the search specification. The search specification includes searchbase, scope, filter, attrs, exattrs, attrsonly, sizelimit, and timelimit parameters as in the normal search specification. The searchbase parameter has no default value and must always be specified. The scope defaults to sub, the filter defaults to (objectclass=*), attrs defaults to "*,+" to replicate all user and operational attributes, and attrsonly is unset by default. Both sizelimit and timelimit default to "unlimited", and only positive integers or "unlimited" may be specified. The exattrs option may also be used to specify attributes that should be omitted from incoming entries.
>The
The schema checking can be enforced at the LDAP Sync consumer site by turning on the schemachecking parameter. If it is turned on, every replicated entry will be checked for its schema as the entry is stored into the replica content. Every entry in the replica should contain those attributes required by the schema definition. If it is turned off, entries will be stored without checking schema conformance. The default is off.
--- >The schema checking can be enforced at the LDAP Sync consumer site by turning on the schemachecking parameter. If it is turned on, every replicated entry will be checked for its schema as the entry is stored on the consumer. Every entry in the consumer should contain those attributes required by the schema definition. If it is turned off, entries will be stored without checking schema conformance. The default is off.
528c528 <The binddn parameter gives the DN to bind as for the syncrepl searches to the provider slapd. It should be a DN which has read access to the replication content in the master database.
--- >The binddn parameter gives the DN to bind as for the syncrepl searches to the provider slapd. It should be a DN which has read access to the replication content in the provider database.
540c540 <This directive is only applicable in a slave (or shadow) slapd(8) instance. It specifies the URL to return to clients which submit update requests upon the replica. If specified multiple times, each
This directive is only applicable in a replica (or shadow) slapd(8) instance. It specifies the URL to return to clients which submit update requests upon the replica. If specified multiple times, each
The next section of the configuration file defines a BDB backend that will handle queries for things in the "dc=example,dc=com" portion of the tree. The database is to be replicated to two slave slapds, one on truelies, the other on judgmentday. Indices are to be maintained for several attributes, and the userPassword attribute is to be protected from unauthorized access.
--- >The next section of the configuration file defines a BDB backend that will handle queries for things in the "dc=example,dc=com" portion of the tree. The database is to be replicated to two replica slapds, one on truelies, the other on judgmentday. Indices are to be maintained for several attributes, and the userPassword attribute is to be protected from unauthorized access.
=== end slapdconfig.html === === begin tls.html === === end tls.html === === begin troubleshooting.html === === end troubleshooting.html === === begin tuning.html === === end tuning.html ===