Problems installing Apps in SharePoint farm

Recently while working with a colleague we came across a few easy to miss issues when configuring the App Management Service in a SharePoint farm and while troubleshooting those issues found a lack of online resources on what turned out to be super-simple issues to resolve. (Hindsight is great)

Issue 1: ‘Let’s try this again’ error installing from the App Store

After installing and configuring the services, app domain and catalog as required, we could now open the SharePoint App Store, and choose an app to install, however the following message was displayed and the installation of any App would never begin.

Capture

Error text: Everything is fine, but we had a small problem getting your license. Please go back to the SharePoint Store to get this app again and you won’t be charged for it.

Everything’s fine hey? Looking at the ULS revealed that this was caused by security:

w3wp.exe (0x0CA0) 0x1DF4 SharePoint Foundation App Marketplace High An exception was thrown while trying to import the app license. AppName:Bulk Edit; Exception:System.UnauthorizedAccessException: You don’t have the right to perform this operation. at Microsoft.SharePoint.SPAppLicenseManager.ImportLicense …
w3wp.exe (0x0CA0) 0x1DF4 SharePoint Foundation App Marketplace High Post app license acquisition did not result in a successful license import. HugState:Retry

But how can this be, we’re testing using the Farm Admin account and so we have all permissions! It turns out, that is actually the problem! While I can’t say specifically where / what setting causes this, in a typical SharePoint 2013 install the Farm Admin (/ Installer account) is intentionally restricted in certain ways, my guess is something to do with local computer policies or such.

Solution

Login using a user account (who can be Site Collection Admin and all that)!

Capture2

Yay my favourite app!

Issue 2: App installs without issue but does not work

Now we have our first app installed, however running it doesn’t quite work out. Unfortunately this time we get very little indication of the problem, in fact with Bulk Edit the app will start and look good, except nothing works! The buttons do nothing, and no data is displayed! :(

Capture3

Similar issues happen with other apps installed and without looking at the F12 browser debug console the only indication that there is a problem is the top right hand corner site icon image which is just a little ‘x’.

If you do debug you will see an error like this:

SCRIPT5009: ‘RegisterSod’ is undefined

File: Default.aspx, Line: 14, Column: 32

Basically that tells me as a developer that something is seriously wrong in the farm (SP.js is missing which the whole JSOM api requires)!

Cause & Solution

Fortunately the solution was simple: SP.js was actually missing! Well not exactly, as it turns out while we had provisioned a single PWA site collection at /PWA in our Web App, we had not actually created a Default Root site collection at ‘/’. This configuration actually can break lots of things including SSRS and so this comes as no surprise really.

Simply creating a root site collection (using Blank or any template) fixed the issue and now all apps work as normal.

 

Other potential issues

While possible the topic of another rather long post, here’s a list of other things to check if you get permissions issues trying to install apps (sorry this is from memory so pls comment below if you think any of this has changed):

 

Hope that helps someone else out there!

SharePoint AppWeb display broken on Office365

This is an issue I have come across deploying Apps in non-English PWA site collections, while the App page loads clearly something is wrong, see a screenshot:

non-endisplay

If you press F12 in IE or Chrome and open the console you will see errors like the following:

Failed to load resource: the server responded with a status of 404 (NOT FOUND) https://*****-1edc3a4f15d104.sharepoint.com/sites/testfi/_catalogs/theme/Themed/634341C6/corev15app-DDE41C8D.themedcss?ctag=0

From my testing this issue only came up in the last few months but fortunately there is a fix.

The problem appears to be caused by the provisioning job not deploying certain resources, fortunately this can be fixed by modifying the Alternate Language settings of the site collection. Simply selecting English as an alternate language will install the required components, and better yet when you then proceed to un-select English from the alternate language options the required resources remain and the issue is fixed for good.

Steps to correct issue

  1. Open Site Settings in your PWA or other site collection where the App is deployed.
  2. Open Alternate Languages, and select English as an alternate language.
  3. Save the settings and wait for the changes to apply.
  4. Now opening the App should work without display issues.
  5. Finally if required repeat step 1 and 2 and de-select English to restore the previous setup (while keeping the fix).

HTH,

MS Reporting (Project Publish) jobs Failed after AprilCU

This is one that has been causing me some grief the past couple of weeks, after apply Project Server 2013 April CU (KB2737990) a couple of customers started reporting the following errors intermittently in the queue:

GeneralQueueJobFailed (26000) – ReportingProjectPublish.ReportProjectPublishMessageEx. Details: id=’26000′ name=’GeneralQueueJobFailed’ uid=’b62dc86f-30cf-e211-9409-00155d11030a’ JobUID=’2cc0c869-30cf-e211-9409-00155d11030a’ ComputerName=…

General Reporting message processor failed: ReportingProjectChangeMessageFailed (24006) – Object reference not set to an instance of an object.. Details: id=’24006′ name=’ReportingProjectChangeMessageFailed’ uid=’beef9169-30cf-e211-9409-00155d11030a’ QueueMessageBody=’Project UID=’e296653a-30cf-e211-9409-00155d11030a’. PublishType=’ProjectPublish” Error=’Object reference not set to an instance of an object.’. …

 

The ULS log indicates the source of this one:

PWA:, ServiceApp:Project Service Application, User:i:0?.t|liveid|…=, PSI: [RDS] ULS Event: ReportingProjectChangeMessageFailed was associated with exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Office.Project.DataEdit.Reporting.ReportingData.
GetTaskBaselineCoreTimephasedDataSetInternal(BaselineEntity[] baselineEntityArray, Int32 nIntervalLengthMinutes, Boolean enforceMaxRowLimit, Int32& index)
at Microsoft.Office.Project.Server.DataAccessLayer.
TaskBaselinesTimephasedDataSync.GetTimephasedDataForEntities(IEnumerable`1 entities, Int32 interval, Int32& index)

 

So something is wrong in the baseline, unfortunately turns out to be nothing small!

Looks like ANY task with 0 duration causes this failure, yes that’s right any milestone included in the baseline!

Microsoft have confirmed this one just today so I certainly won’t be installing AprilCU ever again, however currently no workaround exists, so if like me you are stuck with this then for now take it up with Microsoft as they will hopefully have a fix ASAP, otherwise I’ll update this post when I learn more.

 

HTH,

Update 2/10/2013:

Thanks Carl Dalton for testing and confirming that an old workaround to a similar issue for 2007 actually fixes this one: http://carl-projectserver.blogspot.com.au/2013/09/issues-update-on-projectserver2013.html

Update 16/10/2013:

The Microsoft Hotfix is out in the October 2013 CU, see the actual KB and go download the patch: http://support.microsoft.com/kb/2825659/en-us

ADFS Login Failure on one SharePoint site collection

After migrating a Project Server (or any other SharePoint web app) to a new SharePoint 2010 farm using ADFS for federated authentication and multi-tenancy you receive the below error message when trying to access some of the migrated site collections. In my case the root site worked but /PWA failed.

Also if you reprovision a new site collection (include a PWA) in the same web application it also works fine.

 

Error message from ADFS

clip_image001

There was a problem accessing the site. Try to browse to the site again.

If the problem persists, contact the administrator of this site and provide the reference number to identify the problem.

Reference number: [GUID]

 

Event log on ADFS server

Log Name: AD FS 2.0/Admin
Source: AD FS 2.0
Date: 5/11/2012 10:00:43 PM
Event ID: 364
Task Category: None
Level: Error
Keywords: AD FS

Description:

Encountered error during federation passive request.
Additional Data

Exception details:
Microsoft.IdentityServer.Web.InvalidRequestException: MSIS7042: The same client browser session has made ‘6’ requests in the last ‘5’ seconds. Contact your administrator for details.

at Microsoft.IdentityServer.Web.FederationPassiveAuthentication. UpdateLoopDetectionCookie()

at Microsoft.IdentityServer.Web.FederationPassiveAuthentication. SendSignInResponse(MSISSignInResponse response)

 

Cause

What’s happening is that while authentication works as expected on some sites, when you open a particular site collection the authentication goes into a loop, eventually failing when the ADFS server detects this redirect loop.

After a fair amount of digging the problem turned out to be in the multi-tenancy configuration of this particular farm. Specifically the site collections for some reason did not all have a subscription.

Using the PowerShell command ‘Get-SPSite https://site/sitecollection1’ against two site collections to compare the settings turned up the key difference:

image

 

This makes sense as to why the authentication is failing; the different site subscription overrides the FedAuth authentication cookie assigned by ADFS as soon as it is assigned, causing the page to redirect back to re-authenticate.

 

Resolution

Fortunately once the above was found the fix is simple, run the following command in PowerShell:

Get-SPSite | Set-SPSite 
–SiteSubscription [Guide of site subscription]

 

Now after a quick IISRESET all should be working!

Recovering from Project Server corruption

A customer of mine recently had to recover from a worst case corruption scenario; one in which reverting to a backup was not an option. In short a hardware change resulted in progressive database corruption that was not picked up for over two weeks(!), leaving few options but to attempt to recover the corruption in-place.

Firstly though we had to fix the SQL databases, for that the following command was needed;

alter database ProjectServer_Published SET SINGLE_USER WITH ROLLBACK IMMEDIATE

DBCC CHECKDB(‘ProjectServer_Published’, REPAIR_ALLOW_DATA_LOSS)

alter database ProjectServer_Published SET MULTI_USER;

Yes that parameter does say “Allow Data Loss”, once you run this there is no going back, the databases are now free of consistency errors and other corruptions but in order to reach that state the DBCC command probably had to delete data!

So what now?

Well (fortunately for once!!) Project Server and Project Professional both are no stranger to application level corruption (*cough* 2007 *cough*), so despite DBCC repairing 7000+ consistency errors in the published database alone we are not without options.

So let’s look at some of the errors we had to deal with, firstly the MS Project client side errors and how we dealt with them.

 

MS Project Errors

 

Error 1. There is a circular relationship in task …

image

This one was surprising, but a sign that most likely the last save attempt for this particular schedule wrote corruption to some of the tasks.

To correct this you need to identify the task and correct or delete the predecessors, in my experience the circular reference is pretty obvious so fixing it is quick, however if that’s not possible then I’ve previously found this procedure helpful: http://www.domorethanmanage.com/articles/2008/05/29/Resolvingcircularreferenc.html

 

Error 2. Save Error’s 9000(0x2328) and 26005(0x6595)

clip_image001

clip_image002[6]

I’ve put these two together as the solution is the same, in fact this procedure also applies to the Publish errors that you may get also see.

If you see the above, the first thing to try is an administrative restore, however if that fails there is one last resort;

The “XML Round Trip” method:

1. Open the project in MS Project.

2. File, Save As, Save As File.

3. When prompted select to save with “All Enterprise Custom Fields”.

4. In the file dialog select the Save as type: XML

image

5. Save the file to your local computer.

6. Once saved, close the project (don’t forget to check in!) and reopen the saved XML file, when prompted choose to import ‘As a new project’.

7. Once opened, select Save As and save the project to the project server using the exact same project name.

8. When prompted say yes to overwrite the existing project (you did check it in right?)

9. Publish and your done.

Note; This process will overwrite any previously saved data for that project, the impacts of this are as follows;

  • In-progress timesheets will be affected, remember that internally all tasks and assignments are recreated, so existing OPEN timesheet periods will be updated. This may result in un-submitted timesheets losing actual work, or worse in progress timesheets failing to submit.
  • Reports or custom add-in’s relying on GUID’s will likely have issues.

 

Error 3: Unexpected problem occurred while opening the file

image

If you see this, then hopefully one of the following works, if not you might be out of luck!

  • Open the project from the local project cache.
  • Open the PUBLISHED version of the project by selecting the appropriate Store from the open dialog.

image

  • Restore the project from administrative backup.

 

PWA Errors

Next, a number of errors in PWA can show up as a result of corruption, it’s worth mentioning that pretty much all of the data in PWA comes solely from the Published database. This is good to know because typically 99% of them can be fixed by simply re-publishing!

That’s all good when dealing with a single project, however when your dealing with multiple project corruption, or simply more commonly when a view fails and you don’t know *what* project caused it this method of bulk-publishing will come in handy:

 

Bulk Publishing Projects using ProjTool

Firstly get yourself ProjTool 2010: http://blogs.msdn.com/b/project_programmability/archive/2010/11/03/projtool-for-project-server-2010.aspx

This tool is extremely powerful (IE destructive!) and should be used with caution, but just completing these steps you don’t need to get too deep into the tool so your not risking too much:

1. Open ProjTool, enter project server URL and select Windows for Authentication.

2. Select the projects to republish (multi-select by holding shift).

3. Click Publish from the Toolbar, when prompted select YES for a FULL Publish.

image

From experience selecting over a couple of hundred projects will cause this to fail, but I have often used this to queue up 80+ projects for a republish.

Note however that almost ALWAYS ProjTool will report a problem in the queue, that’s because it only waits for a fixed time period for the queue jobs to complete and that never seems to be enough.

 

PWA View Errors

image

(The view failed to load. Press OK to reload…)

image

(An unknown error has occurred.)

These types of errors are likely caused by one of the following:

  1. Corruption in the project data being viewed
  2. Basic corruption in the view configuration.

For #1 this can be a custom field value that is filtered where one project has an invalid (null?) value for the field in question, so often just removing any filters configured on the view will fix this.

If you identify the filter in question then see the tips above about republishing (or possibly re-saving AND re-publishing) some or all projects to correct the data.

For #2 there are few options other than to recreate the view, copying the current view sometimes helps but not reliably.

 

Project Workspace Permissions Issues

I’m not going to cover how to restore document or general SharePoint data corruption in this blog, but in terms of Project Server usage you’ll most likely have issues relating to to the project permissions synchronisation, for this see the following link: http://nearbaseline.com.au/blog/2011/02/resetting-lost-permissions-in-project-server-2010/

 

Other Issues

Duplication of Custom Field values in PDPs:

image

This one turned out to be just an odd co-incidence in timing see the following recent Microsoft kb article if you see this issue;

http://support.microsoft.com/kb/2598251

 

Finally Reporting Database Refresh

Once all the above has been corrected you may still see some errors in Reporting jobs in the queue;

ReportingProjectChangeMessageFailed (24006) – The INSERT statement conflicted with the FOREIGN KEY constraint

There are a few options available for this problem, two are well described here: http://www.epmpartners.com.au/blog/insert-statement-conflicted-with-the-foreign-key-constraint/

I’d suggest the RDB Refresh (Option 2 in the blog linked), this will take some time (hours typically), and any failures will result in projects NOT being listed in the reporting database at all. This is because the job removes and resynchronises the projects, so typically after running this job I usually use ProjTool to do a bulk publish (see above).

Final Words

The moral of this story is simple: Make sure your SQL DBA regularly uses DBCC CHECKDB (http://www.sql-server-pro.com/dbcc-checkdb.html) to ensure you don’t get into this sort of mess!