Remote Event Receivers – you’re all doing it wrong

Remote Event Receivers are a powerful way to integrate custom code into your SharePoint Online environment.  Essentially a Remote Event Receiver is a hook that allows you to execute your code in response to an event that occurs in SharePoint.  There are several techniques for responding to events in SharePoint Online, but the Remote Event Receiver is the most powerful. It offers dozens of different events to attach to and allows you to configure synchronously or asynchronously. It is also very easy to attach, develop, deploy, test, and maintain. Is also very misunderstood.

Remote Event Receivers were introduced along with SharePoint 2013 and the arrival of the App model. Microsoft provided tooling with Visual Studio to create Remote Event Receivers, but unfortunately the only way to expose this tooling was in the context of a Provider-Hosted App.

This is unfortunate because Remote Event Receivers have nothing to do with Provider-Hosted Apps. In order to develop a Remote Event Receiver using the Microsoft-provided tooling, a developer had to create a Provider-Hosted App project and deploy their RER along with it. This added a great deal of complexity to the development effort, and made packaging and deployment a painful and tedious experience.

To further complicate things, Microsoft decided to wire up a WCF service as the endpoint in its RER tooling. This is sheer lunacy, even back in 2013. A web API project would have been simpler and would be more in line with Microsoft development tooling efforts. The opacity and complexity of WCF made RER development even more cumbersome. Actually, I believe they decided to use the WCF service so the development experience would be similar to that of old-school Event Receivers, with the ability to use a deserialized Event Properties object.

Building a Remote Event Receiver is Easy

In truth, it is remarkably simple to configure, develop, deploy, and maintain a Remote Event Receiver, but in order to do so you must completely abandon the Microsoft tooling and just set up the pieces yourself. Luckily there are really only two components to a Remote Event Receiver:

  • The endpoint
  • The registration

The registration is where you tell SharePoint, “call this endpoint every time this event occurs”. The CSOM provides mechanisms for adding Remote Event Receivers, but the details depend somewhat on the type of event receiver being deployed. The  PnP PowerShell library provides the ability to register a Remote Event Receiver in a one-liner. For example, to set up an RER that is invoked every time an item is updated on a list, execute:

Add-PnPEventReceiver -List "Tasks" -Name "TasksRER" -Url https://my-rer.azurewebsites.net/Service1.svc -EventReceiverType ItemAdded -Synchronization Asynchronous

More about registering Remote Event Receivers

The endpoint is just a web service listening at a certain URL, and you have lots of options for this. A Web API project would work great for this. Azure Functions are also a very compelling option. You are also free to write services in Java, Node or whatever other technology you can think of. In the example below, we’ll use the canonical WCF Service you’d get with the Visual Studio item template, but we’re going to sidestep the template and wire things up ourselves. It’s actually easier this way.

 

Creating the Remote Event Receiver shell

In Visual Studio,

  1.  create an empty ASP.NET web application
  2. Add the Nuget Package ”AppForSharePointOnlineWebToolkit”.
  3. Add a new item of type WCF Service to the web app.
  4. Get rid of the IService reference and set your service to implement IRemoteEventService, which lives in the Microsoft.SharePoint.Client.EventReceivers namespace. This namespace came into the project with the Nuget packages we added earlier. Resolve the squiggly to implement the interface stubs.

Your service class should look something like this:

RER-stub

F5 your project and navigate to the service to make sure it’s accepting requests. Take note of the port number.  We’ll need that to set up our proxy for local debugging.

Locally debugging your remote event receiver

To test this event receiver locally, we’ll use ngrok. According to its documentation, “ngrok is a reverse proxy that creates a secure tunnel from a public endpoint to a locally running web service.”  We will use it to map an Internet endpoint to our local machine so we can intercept and debug requests coming from SharePoint Online.

Assuming you have installed Node.JS and ngrok, create your proxy by executing the following. 56754 is the port number hosting my local WCF service.

ngrok-1

Once it connects it’ll output some data, including the public URL of our proxy connection:

ngrok-2

Next, open up a browser and navigate to the ngrok URL, append the service endpoint, and you should be able to see that the ngrok URL is returning your service.

ngrok-3

 

Next we’ll attach our event receiver to a SharePoint list. Using the PnP PowerShell cmdlet shown above, we’ll add a Remote Event Receiver to our site. Make sure to open a new PowerShell session for this and leave the ngrok session running in its window. The proxy will be released when the window is closed.

Now, set a breakpoint in your ProcessOneWayEvent method, add a task to the list and make an edit to it.  If all goes well your local web service will be called and the breakpoint will hit:

rer-2

Make sure you closely inspect the properties object in the debugger, and get a feel for all the data that’s in there. For this particular event we’ll want to check out ItemEventProperties and ItemEventProperties.AfterProperties for some useful metadata that gets passed into the service.

Deploying your Remote Event Receiver to Azure

When you are ready to deploy to the Internet you can deploy to Azure just like a normal web application. You’ll want to run Add-PnPEventReceiver using the Internet URL to register your event receiver for real, of course.

 

Advertisements

19 thoughts on “Remote Event Receivers – you’re all doing it wrong

  1. Thank you for this article.

    Question tho.
    how can i have the ClientContext?

    my code inside “using (ClientContext clientContext = TokenHelper.CreateRemoteEventReceiverClientContext(properties))”

    is not firing because the properties.ContextToken is null.

    How can i do this?

    Like

      • Hi Derek. Thanks for the article, it was very helpful. I’m wondering if you can shed some light on an issue I’m having. I have a remote event receiver deployed to a Windows Server 2012 box with IIS 8. I have an SP 2013 on-prem install, and I’ve attached an ItemAdded receiver to a library there. I’ve confirmed that the receiver is firing. I’m using your Helpers.GetAuthenticatedContext method to try to grab the context. It seems to blow up for me when it tries to get the context (ClientContext ctx = new ClientContext(siteUrl);). I tried to catch the exception but nothing is showing. Not sure if you have any on-prem experience, but would you know what might be going on here?

        Like

      • Yep, it should work exactly the same in SP 2013. Make sure your site url and credentials are correct. If it’s blowing up at that specific line then I’d guess the URL is improperly formatted. Make sure it’s just the site url with no pages or lists in the url.

        Another thing to consider with on-prem would be firewall issues. Can the event receiver reach out to SharePoint? Do they both exist on the same machine? If not, can you ping the SharePoint server from the event receiver server? Also if the on-prem server has non standard authentication setups like ADFS then it might not work.

        Like

  2. Thanks! The receiver server and the SP server are on different boxes, but I was able to ping SP from the receiver server. I did format the url differently and was able to grab the context. I can get list/list item properties, but now I’m blowing up on the ExecuteQuery method. I’m not seeing a specific error in my logs, but I’m guessing this is a permissions issue, would you agree?

    Like

    • There’s probably an error in your code. Try to get your code working in a console app before porting over to the event receiver, and start with a simple concept, then flesh it out until you have it all working.

      Like

  3. It does work fine in a console app. It’s a very simple action. I have it narrowed down to the ExecuteQuery method. That works fine in the console app, but in the receiver (running on the remote server), it fails.

    Like

  4. We are currently working on a demo project to deploy a python flask application as an SER endpoint. We registered the endpoint with the PnP command as shown in your article but we aren’t receiving any packets. Maybe flask doesn’t recognize the WCF protocol. Our next step is to print out the incoming stream of bytes if there is a problem with registering the SER. Do you have any thoughts on that? Especially swapping out ASP.NET with python.

    Like

    • What you are trying should work just fine. There are no special protocols involved. I’ve used a Microsoft Flow as an event receiver and also a .NET MVC endpoint, and there’s a guy online who did it with Azure Functions. A few troubleshooting steps: Make sure the RER is actually registered and attached to the event you’re expecting (Get-PnpEventReciever or something like that). Next, make sure your Python app is exposed to the internet because the requests are actually coming from SharePoint Online. So if your application is inside a firewall it won’t work. If all that’s wired up you should at least be able to see the traffic coming in. Good luck!

      Like

  5. Hi derek, thanks for your quick reply. We discovered, that it is only possible to wire an event receiver to an endpoint listening on port 80. The problem was that we previously tried to wire the RER to our endpoint listening on port 5000, ofc publicly available. Tested with SharePoint Online.

    Sadly SharePoint Online communicates via SOAP. I don’t come from a microsoft background but I read that WCF 4.0 (https://en.wikipedia.org/wiki/Windows_Communication_Foundation#Interoperability) supports the JSON format compared to this non-modern XML syntax. Do you know if it is possible to force SharePoint Online communication via JSON instead of XML?

    If not the next logical step for us is to parse the SOAP XML Envelope. Do you know any (parsing?) libraries (doesn’t matter if .NET based or not) which make life easier? Thanks for your help.

    Like

    • Yeah, the port 80 restriction makes sense but I never really though about it. There’s no way to make SharePoint change the way it posts its RER data but I’d love to be proven wrong on that. WCF/XML was in interesting architectural decision for sure. As far as parsing goes, I’d probably use XmlSerializer in .NET or xml2js in Node. I wouldn’t know what to do in Python. You might also want to look into WebHooks if you haven’t already.

      Like

      • Yeah, I already looked into webhooks. Looks more developer friendly… but we would like to leverage the ProcessTwoWayEvent, basically the sync functionality of the RER. So there is no other choice.

        Actually, my company is using SharePoint since ~2000 and I wrote/maintain the one or other farm solution for SP 2010/2013. We have multiple customers using different customer-specific farm solutions. Microsoft’s focus will clearly be at its cloud business in the future. It’s a matter of time customers have to update their 2010/2013 on-premise versions due to the EOL. Some of our customers even have SharePoint Online + the on-premise version with their farm-solutions.

        Over the last years, it felt tedious developing SP solutions. So I convinced my boss to create new projects from scratch with other, open source technologies. Basically, a 12factor distributed system with containerized services – a docker/linux/python/django/rest/nginx/postgres stack. Oh boy, what a dream compared to the tedious work years before… while working on such projects the wind changed: microsoft announced msql/pwsh for linux, .NET core, etc. And now that time has come and even my boss questions and fears the future of on-premise/farm solutions we are evaluating SharePoint Online programming models. In my opinion finally a step in the right direction.

        Ofc, I am a bit biased because of my work with declarative frameworks in the python universe and the established CI/CD workflow we now have in other projects. So my expectations are high. But what a disappointment. The SharePoint/PnP-PowerShell module is installable on linux, but throws an error when connecting, there are no official libraries for python/php/node, the CSOM libraries are not .NET core compatible ( there is a work in progress badge on the feature request site), the last time someone commented on the PnP issue list was 3 weeks ago, I am dealing with 404 links in the docs, the collapsible navigation menu on msdn gives me ulcer :(, the majority of blog post (yours seems to be an exception) uses screenshots like click here and click there without any explanation. I don’t want to complain too much. I just want to find a good compromise between the SharePoint world, which has a good OOTB functionality, and a developer’s vivid needs.

        The future clearly points to the cloud and it’s a matter of time some of our customers decide to abandon on-premise versions. Right now it seems that the only hassle free option is to use SharePoint’s REST API. For other functionality we have to use the RER, lets see how my demo projects envolve in the future. But when I think about that I have to RDP into a windows machine to interact with my sharepoint online resources and browse around not well maintained official repositories at github I already start to question again if this is really the way to go and how the future should look like. Thanks for your help and sorry for my grammar.

        Like

  6. Hi Derek, I’ve managed to publish my RER to azure and attached it to my SPO project, it works fine when im using the http protocol.

    but when we host it in our server that is using https it doesn’t work as expected. do you have any experience on this?

    Thanks!

    Like

    • Make sure the RER is registered using an https url. Assuming you’ve done that it sounds like a misconfiguration or certificate issue on your server. But assuming it’s reachable from the Internet it should work. What do you mean, it “doesn’t work”? Your’re getting an error? What does it say?

      Like

  7. It is reachable on the internet via https and i registered it in my list using the https url as well. “doesn’t work” by means of it seems that its not getting trough the ProcessOneWayEvent because i didn’t see the changes i expect.

    i’ve also checked our IIS logs and see “2018-12-11 05:33:20 POST /addin/autotag/pbpo_autotag.svc – 443 – – – 404 0 0 206”

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s