Modernizing SharePoint with new libraries

The Microsoft 365 PnP Community project has been around for more than six years already, and since the very beginning has provided open-source libraries and toolkits to help developers reach their goals easily and comfortably. Some of the most famous and useful libraries are:

  • PnP Sites Core: to improve productivity and quality of life of .NET Framework developers. It includes the “famous” PnP Provisioning Engine.
  • PnP PowerShell: to improve productivity leveraging hundreds of helpful cmdlets.

Every month, almost 80K tenants use these libraries to facilitate approximately 34B requests toward Microsoft Graph and the Microsoft 365 REST APIs. That’s indeed a massive amount of usage! However, in the last few years, the development world has changed and improved, as has Microsoft 365.

Microsoft introduced the “modern” UI in almost all of its cloud services, as well as new workloads and services. For example – think about the rise of Microsoft Teams, the new Microsoft Lists, and many more services.

Nowadays, we see a clear demand for cross-platform development and supporting the “modern” UI as well as new emerging services. That’s why Microsoft 365 PnP has been working on renewing the whole offering of Microsoft 365 PnP developer libraries since early 2020. In Figure 1, you can see the roadmap that we have been using as a reference:

How the new wave of libraries from Microsoft 365 PnP will change the way you modernize SharePoint

The main goal of this new wave of libraries is to support cross-platform development, introducing support for Microsoft .NET Standard 2.0 and Microsoft .NET 5.0. Additionally, the new libraries will provide the needed tooling to easily modernize your classic solutions so you can create new modern solutions.

The Microsoft 365 PnP Community project also improved the development process of these new libraries. There is now fully automated testing for all contributions from the community, in addition to nightly builds of all libraries to speed up the adoption of changes and release of new features. Microsoft 365 PnP will continue to roll out major releases concerning the libraries, but in the meantime, you can also install the latest NuGet package of last night’s build, if you are interested in riding the waves and use a preview of the latest bits.

Watch on-demand: 7 Reasons to Upgrade to SharePoint 2019 or SharePoint Online and Move to Modern UI

PnP Framework

The PnP Framework was released as v. 1.0 on January 2021 and is the evolution of PnP Sites Core, which is now retired and archived. The new PnP Framework targets .NET Standard 2.0 and .NET 5.0, so you can use it on any platform (Windows, Linux, MacOS) and in any cloud-based service like Azure App Services, Azure Functions, Containers, etc.

By design, with this new library, we decided to support SharePoint Online (SPO) only. As such, if you need to target SharePoint on-premises (2013/2016/2019), you should still rely on PnP Sites Core. However, keep into account that PnP Sites Core will not evolve anymore. On the contrary, if you are developing a modern solution for SPO using the PnP Sites Core library, you should definitely remove it and replace it with the new PnP.Framework NuGet package.

From a functional point of view, the new PnP Framework is fully backward compatible with PnP Sites Core, but it is cross-platform and uses the new Client Side Object Model (CSOM) of SPO for .NET Standard. We mainly target modern authentication with OAuth 2.0, but we still provide support for ACS based authentication for apps in app-only mode. In the following code sample, you can see how to use the PnP Framework to connect to SPO and consume some data, using an instance of the AuthenticationManager class. By default, PnP Framework uses a multi-tenant Azure Active Directory application called PnP Management Shell, which you should register using PnP PowerShell.

// Create an instance of the AuthenticationManager type
var authManager = new AuthenticationManager(username, password);
// Get a reference to the ClientContext of CSOM
using (var context = await authManager.GetContextAsync(siteUrl))
{
    // Read web properties
    var web = context.Web;
    context.Load(web, w => w.Id, w => w.Title);
    await context.ExecuteQueryRetryAsync();

    Console.WriteLine($"{web.Id} - {web.Title}");

    // Retrieve a list by title together with selected properties
    var documents = web.GetListByTitle("Documents", l => l.Id, l => l.Title);

    Console.WriteLine($"{documents.Id} - {documents.Title}");
}

As you can see, the library supports asynchronous programming and provides a set of extension methods and helper types to extend what is offered out of the box by the CSOM library of Microsoft.

To access a target SPO site, you simply need to create an instance of the AuthenticationManager type, using any of the supported authentication techniques like:

  • Interactive user authentication
  • Username and password (as you can see in the previous code sample)
  • Device code with an application registered in Azure Active Directory
  • Authorization Code flow
  • Client ID and Client Secret for app-only authentication toward Microsoft Graph
  • X.509 Certificate for app-only authentication
  • On-behalf-of flow
  • Azure ACS for app-only

Once you have created the instance, you can get a reference to the CSOM ClientContext through the AuthenticationManager instance, and you can consume SPO. You can always leverage tons of extension methods provided by PnP Framework, rather than reinventing the wheel and making a lot of manual work to consume SPO. For example, in the code sample above, you can see the GetByTitle extension method, which retrieves a list instance by title, including a list of properties selected through a set of lambda expressions, simply with one line of code. This would be akin to ten lines of code using plain CSOM. Another example is the ExecuteQueryRetryAsync extension method, which takes care of executing a CSOM query with a built-in retry logic that will help you manage throttling requests, if any.

In the PnP Framework, we also provide the implementation of the PnP Provisioning Engine, which is the engine used by many businesses as well as Microsoft itself, to automate the provisioning of sites and artifacts. For example, the PnP Provisioning Engine is behind the scenes of the Microsoft Look Book site (http://lookbook.microsoft.com/), as well as behind the scenes of the provisioning of SPO demo content in Microsoft 365 developers’ tenant.

Last but not least, the PnP Framework is in the back-end of most of the cmdlets provided by the PnP PowerShell cmdlets and provides the basic infrastructure for the Page Transformation engine of the PnP Modernization Framework.

Generally speaking, the new PnP Framework is the “de facto” library that every SharePoint developer should use to create modern solutions for SPO in Microsoft.NET.

PnP Core SDK

The PnP Core SDK is a fresh new library that was introduced for modern cloud development and is going live in early February 2021. It targets .NET Standard 2.0 and .NET 5.0, so you can use it on any platform (Windows, Linux, MacOS) and in any cloud-based service. Additionally, it is available as a NuGet package with the name PnP.Core.

The PnP Core SDK library has been designed with modern development in mind. Microsoft natively supports Dependency Injection (DI), a service-oriented architecture; fluent syntax including support for Language Integrated Query (LINQ) as well as an asynchronous development model. Microsoft also natively supports requests batching at REST level and automatic retry logic in case of requests throttling.

You might be wondering what the difference is between PnP Framework and PnP Core SDK. The PnP Core SDK is used by the PnP Framework and it’s been implemented with a mindset that is independent of CSOM of SharePoint. More and more new functionalities introduced in SPO are not necessarily available through CSOM; this was the impetus for Microsoft 365 PnP introducing a new SDK, which is under the cover of both PnP Framework and PnP PowerShell for some functionalities.

Internally at Microsoft 365 PnP, this was referred to as the PnP Graph First library, because the is to help modern .NET developers consuming SPO and Microsoft Teams (Teams) via Microsoft Graph. Microsoft Graph is THE standard API for consuming any of the Microsoft 365 workloads, including SPO and Teams. However, at the time of this writing, there are some functionalities that are not yet available through Microsoft Graph, such as support for modern pages, or functionalities that require improvements to be fully comparable with what is offered by the REST API of SPO.

With PnP Core SDK we provide an abstraction layer on top of the back-end APIs so that developers can focus on their main business goal: creating modern solutions on top of SPO and Teams. Undercover, the PnP Core SDK will target Microsoft Graph if the functionality that the developer is looking for is available through Graph, or it will fall back to the REST API of SPO.

It is widely understood that the Microsoft Graph is rapidly growing and evolving, and by using this technique developers won’t need to continuously update their code. They can simply rely on PnP Core SDK, which will make the right choice for them. If a functionality is missing today in Microsoft Graph, when it is eventually added developers will simply need to update the PnP Core SDK NuGet package, without the need to change any line of their code.

The PnP Core SDK targets cloud workloads only (SPO and Teams), and from an authentication point of view, it relies on an open model based on OAuth 2.0. Basically, you can implement the authentication layer with whatever technique you like, as long as you provide a valid Access Token. In case you would like to use the Microsoft Authentication Library (MSAL), we provide a dedicated package, called PnP.Core.Auth, that you can plug-in and use to authenticate access to the target cloud services through MSAL.

In order to use the new PnP Core SDK you simply need to create a .NET project. Then add a reference to the NuGet package with the name PnP.Core.Auth, which includes a dependency on the main PnP.Core package, and you can start using it. In the following code excerpt, you can see how to access SPO in a .NET Console application, configured to use DI through the Microsoft.Extensions.Hosting NuGet package, with interactive authentication:

var host = Host.CreateDefaultBuilder()
// Configure services with Dependency Injection
.ConfigureServices((hostingContext, services) =>
{
    // Add the PnP Core SDK library services
    services.AddPnPCore();
    // Add the PnP Core SDK library services configuration from the appsettings.json file
    services.Configure<PnPCoreOptions>(hostingContext
        .Configuration.GetSection("PnPCore"));
    // Add the PnP Core SDK Authentication Providers
    services.AddPnPCoreAuthentication();
    // Add the PnP Core SDK Authentication Providers 
    // configuration from the appsettings.json file
    services.Configure<PnPCoreAuthenticationOptions>(hostingContext
        .Configuration.GetSection("PnPCore"));
})
// Let the builder know we're running in a console
.UseConsoleLifetime()
// Add services to the container
.Build();

// Start console host
await host.StartAsync();

// Optionally create a DI scope
using (var scope = host.Services.CreateScope())
{
    // Obtain a PnP Context factory
    var pnpContextFactory = scope.ServiceProvider
        .GetRequiredService<IPnPContextFactory>();
    // Use the PnP Context factory to get a PnPContext for the given configuration
    using (var context = await pnpContextFactory.CreateAsync("SiteToWorkWith"))
    {
        // Retrieving web with lists and masterpageurl loaded ==> SharePoint REST query
        var web = await context.Web.GetAsync(p => p.Title, p => p.Lists,
        p => p.MasterUrl);

        // Work with retrieved objects ...
    }
}

// Cleanup console host
host.Dispose();

As you can see, the PnP Core SDK provides a context object of its own, called PnPContext, that you can use to access SPO or Teams using a dedicated Domain Model of types, which is fully independent of CSOM or Microsoft Graph, and which provides the abstraction layer needed to decouple the PnP Core SDK from the back-end REST services. You can also notice the fluent syntax, where you can access the Web property of the PnPContext instance and invoke methods on it, without the need to load the web object instance first.

To execute a sample like the one you just saw, you need to provide a configuration file (appsettings.json) like the one you see in the following code excerpt:

{
  "PnPCore": {
    "Credentials": {
      "DefaultConfiguration": "interactive",
      "Configurations": {
        "interactive": {
          "Interactive": {
            "RedirectUri": "http://localhost"
          }
        }
      }
    },
    "Sites": {
      "SiteToWorkWith": {
        "SiteUrl": "https://contoso.sharepoint.com/sites/pnp"
      }
    }
  }
}

If you like, you can also provide the configuration settings simply using code, without relying on a .json configuration file. You can find further details about how to use the new PnP Core SDK on the official documentation site (https://pnp.github.io/pnpcore/).

Regardless of the technique that you will use to create a configured instance of the PnPContext type, once you have it, you can easily access the SPO site related to the context. In case the site is a modern Team Site, you can also access the Team in Microsoft Teams that is connected to the Team Site, like in the following code excerpt:

// Get a reference to the team connected to the current site, including the FunSettings
var team = await context.Team.GetAsync(t => t.FunSettings);

// Show one of the settings in the FunSettings property
Console.WriteLine($"Are Giphy allowed? {team.FunSettings.AllowGiphy}");

Another interesting thing that you can do is to use LINQ to query the object model of PnP Core SDK. For example, in the following code excerpt, you can see how to query the lists of the current web object sorting the result by title from Z to A:

// Define a LINQ query on lists of the current web
var lists = (from l in context.Web.Lists
            orderby l.Title descending
            select l);

// Browse the lists resulting from the LINQ query
foreach (var list in lists)
{
    Console.WriteLine($"{list.Id} - {list.Title}");
}

These short examples should provide you with a basic understanding of the PnP Core SDK potential, and how helpful it can be while building modern solutions in SPO.

PnP PowerShell

Most of the SharePoint Developers and IT Professionals use the PnP PowerShell cmdlets daily to manage their SharePoint environments, provision artifacts and migrate content or pages. At the time of this writing, the PnP PowerShell library for Windows PowerShell targeting SPO has been downloaded more than 5M times. However, the PnP PowerShell for Windows PowerShell is now archived and retired.

In fact, in January 2021 Microsoft 365 PnP released a new version of PnP PowerShell, which targets PowerShell Core and works on any platform (Windows, Linux, MacOS), including the Microsoft Azure Cloud Shell. Moreover, it works great in an Azure Function that can be triggered by a flow in Power Automate or by a webhook, or a timer trigger, etc. Of course, if you target SharePoint on-premises, you can still rely on the legacy one, however, it will not evolve anymore.

As it happens with PnP Framework and PnP Core SDK, the new PnP PowerShell targets SPO only and mainly supports OAuth 2.0 authentication, even if there are some edge cases where the “legacy” browser-based authentication is still supported.

To upgrade your system and target the new PnP PowerShell, first, you will need to install PowerShell Core and eventually the Windows Terminal app if you are on a Windows machine. Then you can remove the legacy PnP PowerShell executing the following command in Windows PowerShell:

Uninstall-Module -Name "SharePointPnPPowerShellOnline" -AllVersions -Force

Lastly, in the new PowerShell Core console, you can execute the following command:

Install-Module -Name "PnP.PowerShell"

It will install the new library and set you ready to experience the future of PnP PowerShell. Mind the fact that some of the command lets got renamed during the transition from Windows PowerShell to PowerShell Core. Here (https://pnp.github.io/powershell/articles/upgrading.html) you can find the whole list of breaking changes.

From a functional point of view, the new PnP PowerShell provides the same main functionalities of the legacy library, eventually with some renamed cmdlets, plus some new capabilities in the fields of Microsoft Teams management, Microsoft Power Automate flows management, etc.

Again, if you plan to create modern solutions with SPO, the new PnP PowerShell is a must-have. Just for the sake of giving you an overview of the potentials of the new library, in the following script excerpt, you can see how to connect to an SPO tenant and modernize a page from a classic site to a modern site, using under the cover the PnP Modernization Framework that is part of the PnP Framework library:

Connect-PnPOnline "https://<tenant>.sharepoint.com/sites/PublishingPortal"
ConvertTo-PnPPage -Identity "Article.aspx" -PublishingPage -TargetWebUrl "https://<tenant>.sharepoint.com/sites/ModernCommunicationSite" -Overwrite

You simply need to connect to the source site using Connect-PnPOnline. The target site could be for example a classic site based on the Publishing Portal template, and using the ConvertTo-PnPPage command let you can transform and modernize a classic publishing page into a new modern page of a Communication Site. This is just the tip of the iceberg, and in future articles, we will dig into the potential of the PnP Modernization Framework, which is under the process of being migrated from the PnP Framework library to the PnP Core SDK library to make it independent from CSOM and fully based on Microsoft Graph and SPO REST APIs.

What to use when?

After reading this content, you might ask yourself: “What should I use and when?”

If you are a SharePoint developer, used to working with CSOM, or if you have an already existing project built on top of PnP Sites Core and CSOM, you should rely on PnP Framework and start building modern solutions leveraging the rich set of extensions and utilities provided by the new PnP Framework library.

If you are a .NET developer, willing to create a new modern solution for SPO, start working with PnP Core SDK and enjoy consuming the Microsoft Graph and the SPO REST APIs with a high-level abstraction layer.

If you are any of the above, or if you are an IT Professional, download and install PnP PowerShell for PowerShell Core and start using it for any daily maintenance tasks – building your own scripts, managing your SPO environments, etc.

For more information from industry leaders around SharePoint migration and modernization, check out this on-demand webcast.  

Comments

  1. Philippe

    thank you, I get a message stating “Authentication complete. You can return to the application. Feel free to close this browser tab.”, then the program crashes: “using (var context = await pnpContextFactory.CreateAsync(“SiteToWorkWith”))”
    with error message: “‘SharePoint Rest service exception'”
    I am at my wits end on why?
    i have noticed that the HTTP requests that was sent appends: _api/Web?$select=Id%2cUrl%2cRegionalSettings%2f*%2cRegionalSettings%2fDateFormat&$expand=RegionalSettings%2cRegionalSettings%2fTimeZone to the end. and I don’t believe that the Sharepoint location has hat structure. How do I avoid that?

  2. Jithil

    Where to put the configuration code in a .NET Web API project ?

  3. furtivesock

    Hello,
    Thanks a lot! I was lost with all of those changes so this article helped me to understand them better.

Leave a Reply