Get your most visited pages in a serverless way

– Dalibor Mesarić

Get your most visited pages in a serverless way

I was asked what would be a good way to track top read articles for a clients web site. The web site is build using Episerver CMS, so should we save page visits in the database? Or is there a more modern solution?

I knew we already have Application Insights monitoring in place, so we know what are the top visited articles. We just needed to extract that information and put it into code we could use to build such a widget.

This gave me an idea and I wanted to try it out in my own blog.

Setup

I set up Application Insights in Azure according to this documentation. I modified my hexo theme and included the Application Insights script.

The documentation also explains how to use Kusto Query Language to extract data from the logs.

Application Insights is tracking page views, so here is the KQL query I used to get page views for my blog posts. Since all of the blog posts start with /YYYY/MM/DD/, I used regex to fetch only those pages. 

1
2
3
4
5
pageViews
| where timestamp > ago(7d)
| where operation_Name matches regex @"^(\/)\d{4}(\/)\d{2}(\/)\d{2}.*$"
| summarize visits = sum(itemCount) by url 
| top 10 by visits desc

logs

Next step was to get that data through the Application Insights API. I followed the instructions documented here to get an API key, and I used PowerShell to get the initial data from the API.

1
2
3
4
$appid = "Application ID here..."
$apikey = "API key here..."
$response = Invoke-WebRequest "https://api.applicationinsights.io/v1/apps/$appid/query?query=pageViews | where timestamp > ago(7d) | where operation_Name matches regex @'^(\/)\d{4}(\/)\d{2}(\/)\d{2}.*$' | summarize visits = sum(itemCount) by url | top 10 by visits desc" -Headers @{"X-Api-Key"="$apikey"} -UseBasicParsing
$response.Content

content

Now that I got a JSON response, I used it to generate C# class using this online tool.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public class Column
{
    public string name { get; set; }
    public string type { get; set; }
}

public class Table
{
    public string name { get; set; }
    public List<Column> columns { get; set; }
    public List<List<object>> rows { get; set; }
}

public class RootObject
{
    public List<Table> tables { get; set; }
}

Finally I combined all from the above into a simple Console Application.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Program
{
    private const string URL = "https://api.applicationinsights.io/v1/apps/{0}/{1}?{2}";
    private const string appid = "Application ID here...";
    private const string apikey = "API key here...";

    static async Task Main(string[] args)
    {
        HttpClient client = new HttpClient();
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        client.DefaultRequestHeaders.Add("x-api-key", apikey);
        var request = string.Format(URL, appid, "query", "query=pageViews | where timestamp > ago(7d) | where operation_Name matches regex @'^(\\/)\\d{4}(\\/)\\d{2}(\\/)\\d{2}.*$' | summarize visits = sum(itemCount) by url | top 10 by visits desc");
        HttpResponseMessage response = await client.GetAsync(request);

        var rootObject = await response.Content.ReadAsAsync<RootObject>();

        int i = 0;
        rootObject.tables.Single().rows.ForEach(row => Console.WriteLine($"{++i}. URL: {row[0]}, Visits: {row[1]}"));
    }
}

console

Next steps

I have some ideas where to go next. I would put my code in an Azure Function, so that it can be maintained separately from the main application and accessed on demand.

For Episerver implementation I would suggest using a Scheduled Job to call the Azure Function and update the Most Visited Articles widget.

As far as my blog is conserned, here are some possible solutions:

  • Call the API on each page load to show the most fresh data
  • Integrate the API call on each deploy, and generate a static widget
  • Set up a scheduled deployments that would update this widget regularly

Honestly this blog doesn’t have a lot of views, so maybe it is not worth the effort, but it would be a fun exercise. As an experiment I will share this blog post on my newly created dev.to page and my LinkedIn for the first time, along side my usuall Twitter shares. Let’s see if the networking effect works :)

To take screenshots I use LightShot