SharePoint: The complete guide to user profile cleanup – Part 3 – 2013
This is part 3 in a series. You can find other parts here:
SharePoint: The complete guide to user profile cleanup – Part1
SharePoint: The complete guide to user profile cleanup – Part 2 – 2010
SharePoint: The complete guide to user profile cleanup – Part 4 – 2016
SharePoint: The complete guide to user profile cleanup – Part 5 – 2019
Update 12/8/19: Found that deletions are processed differently for a partitioned user profile service app. See the ** amendment in the “ADI Step 2” section below.
Sync Options:
In SharePoint 2013, you have two options. You can use “SharePoint Profile Synchronization” AKA: “FIM Sync”. This is where we use a custom build of Forefront Identity Manager 2010 (FIM) built into SharePoint 2013 to sync user profiles. You can also use another option that was introduced in SharePoint 2013 called Active Directory Import (aka: “AD Import”, “ADI”). For some details about the differences and switching between the two import types, see my previous post here: https://joshroark.com/sharepoint-considerations-when-switching-from-fim-sync-to-ad-import/
Active Directory Import (aka: ADI)
ADI Step 1: Determine if the profile is already marked for deletion.
Run this SQL query against the Profile database:
If your target profiles are in the results, that means they are already marked for deletion. All you should need to do is run the My Site Cleanup Job.
Note: Managed profiles marked for deletion should also show in Central Admin | Your UPA | Manage User Profiles | Profiles Missing from Import.
ADI Step 2: Run a Full Import.
“Out of Scope” (disabled, filtered, moved to a non-imported OU) users do not have their profiles automatically cleaned up by an incremental import**. With AD Import, we don’t use the Sync database to store “state” information about each user. As such, the only way AD Import can tell if a user has fallen “out of scope” is to import them. If the user object has not changed in AD, an incremental import will not pick them up. Luckily, AD Import is fast, so running a Full Import is not a big deal. For more on this, see my colleagues post on the subject: https://blogs.msdn.microsoft.com/spses/2014/04/13/sharepoint-2013-adimport-is-not-cleaning-up-user-profiles-in-sharepoint-whose-ad-accounts-are-disabled/
Note: By default, SharePoint only runs Incremental imports on schedule. In order to run a Full import on a schedule, you’ll have to create a scheduled task and kick it off with PowerShell like this:
$UPA = Get-SPServiceApplication | ? { $_.typename -match “profile”}
$UPA.StartImport($true)
** The exception to this rule is if you directly delete AD users from a synched OU (one that is selected for import). In that case, the deletion of the profile is processed automatically during the next import. You’d see entries like this in the ULS log during the import (if your logging is Verbose):
11/19/2019 14:05:47.48 OWSTIMER.EXE (0x1F34) 0x1098 SharePoint Portal Server User Profiles aei5q Verbose QueueItemChange: Incoming delete for item <GUID=c5b7cfb9-d98d-437a-acc5-da1bc6453a4c>;<SID=S-1-5-21-1700552430-3460358242-3531541990-1701>;CN=User1\0ADEL:c5b7cfb9-d98d-437a-acc5-da1bc6453a4c,CN=Deleted Objects,DC=contoso,DC=com of type 0. 2377199f-f0a8-e088-a10e-e2e15e2daf5a
11/19/2019 14:05:47.51 OWSTIMER.EXE (0x1F34) 0x1098 SharePoint Portal Server User Profiles c8hz Verbose ProfileImportExportService.UpdateWithProfileChangeData: Begin Delete CN=User1\0ADEL:c5b7cfb9-d98d-437a-acc5-da1bc6453a4c,CN=Deleted Objects,DC=contoso,DC=com 9c570013-54d5-4ab2-8097-8f90d6985a0b
11/19/2019 14:05:47.54 OWSTIMER.EXE (0x1F34) 0x1098 SharePoint Portal Server User Profiles c8i0 Verbose ProfileImportExportService.UpdateWithProfileChangeData: End Delete CN=User1\0ADEL:c5b7cfb9-d98d-437a-acc5-da1bc6453a4c,CN=Deleted Objects,DC=contoso,DC=com 9c570013-54d5-4ab2-8097-8f90d6985a0b
However, this is only true if your User Profile Service Application is the regular non-partitioned variety. If you have a partitioned UPA, these profile deletions for deleted AD users do not occur automatically. In that case, you need to deal with them just like other “out of scope” profiles. You can check if your UPA is partitioned or not using this PowerShell:
$upa = Get-SPServiceApplication | ? {$_.typename -match “profile”}
$upa.Properties |fl
ADI Step 3: Mark non-imported profiles for deletion.
Run the following PowerShell to get a list of all your unmanaged profiles:
$upa = Get-spserviceapplication | ?{$_.typename -match “profile”}
Set-SPProfileServiceApplication $upa -GetNonImportedObjects $true | out-file c:\temp\NonImportedProfiles.txt
If the target profiles show up in the “NonImportedProfiles.txt” file, then you need to manually mark them for deletion with PowerShell:
$upa = Get-spserviceapplication | ?{$_.typename -match “profile”}
Set-SPProfileServiceApplication $upa -PurgeNonImportedObjects $true
If the target profiles are managed profiles, not marked for deletion, and you have run a Full Import, then you need to look into why AD Import is not marking them for deletion.
Document your connection filter and selected OUs / containers and check your target profiles against them. If you’re using a complex LDAP filter on your import connection, you should consider using an LDAP tool like LDP.exe or LDAP Browser to test the LDAP filter and make sure it includes and excludes the users you think it should.
ADI Step 4: My Site Cleanup Job
While “Set-SPProfileServiceApplication $upa -PurgeNonImportedObjects $true” marks out-of-scope profiles for deletion, it doesn’t actually delete anything. That’s left to the My Site Cleanup Job.
Check Central Administration | Monitoring | Timer Jobs | Review Job Definitions | My Site Cleanup Job. Make sure it’s set to run at least once per day (default in SharePoint 2013 is once daily).
If the target user profiles are marked for deletion (bDeleted =1), and the Mysite Cleanup timer job is running, but the profiles are not being deleted, then there is some problem with the timer job. You should review the SharePoint ULS logs from the server that ran the job, covering the timeframe when the job ran.
Profile Synchronization (aka: “FIM Sync”)
Note: This section is identical to my SharePoint 2010 post because with FIM Sync, there is no difference in profile cleanup between the two versions: SharePoint: The complete guide to user profile cleanup – Part 2 – 2010
FIM Sync Step 1: Determine if the profile is already marked for deletion.
Run this SQL query against the Profile database:
If your target profiles are in the results, that means they are already marked for deletion. All you should need to do is run the My Site Cleanup Job.
Note: Managed profiles marked for deletion should also show in Central Admin | Your UPA | Manage User Profiles | Profiles Missing from Import.
FIM Sync Step 2: Determine if the profile is managed or unmanaged.
Run the following PowerShell to get a list of all your unmanaged profiles:
$upa = Get-spserviceapplication | ?{$_.typename -match “profile”}
Set-SPProfileServiceApplication $upa -GetNonImportedObjects $true | out-file c:\temp\NonImportedProfiles.txt
If the target profiles show up in the “NonImportedProfiles.txt” file, then you need to manually mark them for deletion with PowerShell:
$upa = Get-spserviceapplication | ?{$_.typename -match “profile”}
Set-SPProfileServiceApplication $upa -PurgeNonImportedObjects $true
If the target profiles are managed profiles and not marked for deletion, then you need to look into why the Sync is not marking them for deletion.
Document your Sync connection filters and selected OUs / containers and check your target profiles against them.
Take a look a the FIM Client (miiscleint.exe) on the server running the Synchronization service. Detailing exactly what to look for in the FIM client is beyond the scope of this blog post, but generally speaking, if you have entire Sync steps that are failing, that’s likely the problem.
FIM Sync Step 3: Run a Full Sync.
If you’ve made recent changes to your Sync connection filters or AD container selection, it takes a Full Sync to apply those changes to all profiles. Also, an Incremental Sync only gets one shot at updating a profile. If something went wrong during the Incremental that ran right after the user fell out-of-scope (deleted from AD, etc), that change is missed. If the user object in AD does not change again, the Incremental will not attempt to pull that user in again. Therefore, a failure during a single run of the Sync could cause the profile to never be processed. For this reason, we recommend that you run a Full Sync on some type of recurring schedule. The interval is up to you, but something between once a week and once a month should work. There is no way to schedule a Full Sync in the UI, but you can accomplish the same thing with a Windows Scheduled Task and this PowerShell:
$siteUrl=”http://yourCentralAdminSiteHere/” #Any site associated with target UPA
$site= New-Object Microsoft.SharePoint.SPSite($siteUrl)
$serviceContext = [Microsoft.SharePoint.SPServiceContext]::GetContext($site)
$configManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileConfigManager($serviceContext)
$configManager.StartSynchronization($true)
If the target profiles have been deleted in Active Directory, but the Sync is not marking them for deletion, the Active Directory Recycle bin may be in play as documented here:
FIM Sync Step 4: My Site Cleanup Job
While the Sync marks out-of-scope profiles for deletion, it doesn’t actually delete anything. That’s left to the My Site Cleanup Job.
Check Central Administration | Monitoring | Timer Jobs | Review Job Definitions | My Site Cleanup Job. Make sure it’s set to run at least once per day (default in SharePoint 2013 is once per day).
If the target user profiles are marked for deletion (bDeleted =1), and the Mysite Cleanup timer job is running, but the profiles are not being deleted, then there is some problem with the timer job. You should review the SharePoint ULS logs from the server that ran the job, covering the timeframe when the job ran.