Trust no one: especially about SharePoint Search

Some days ago, the customer said he want to be able to search files in OneDrive using O365 search.
The solution is there: https://technet.microsoft.com/en-us/library/dn627526.aspx
After Step 5, i went into the search site, selecting OneDrive result source, and i run the search with no parameter... wow only OneDrive results; after that i tried another search with a word and... surprise surprise... results from everywhere.
After lot of hours of tentatives, the decision was to filter results again also in the query text of the search web part (path:https:// tenant_name -my.sharepoint.com/personal), in order to mantain mental sanity.

Read More

Customize search results "without" display templates in O365

I know, the blog post is provocatory, but, i mean, if you have to modify something on default search results you'll probably say:
"hey this is a no problem, we can modify display templates"
Ok..ok that's true, but if i have to inject something in all results display templates, you'll probably know there are tons of those display templates, so you would prefer not to touch those templates.
Sometimes you got to take the left-hand-way, so i noticed in the search subsite, that O365 load this javascript namespace Srch.
I've also found an article where the author did something similar (but i cannot find it anymore).

function hookScript() {
    if (typeof Srch !== "undefined" && typeof Srch.Result !== "undefined") {

  // Save the original function somewhere else
        Srch.Result.prototype.originalRender = Srch.Result.prototype.render;
  
  // Substitute the original function with something more comples
        Srch.Result.prototype.render = function () {
            
   // Here we can do something before rendering results
   
            this.originalRender();

   // Here we can add custom code as we wish after results render
   // In the example below i cycle amond search results doing things
            var results = jQuery(".ms-srch-item-path");

            jQuery.each(results, function(index, value) {
                ...
            });
        };
    };
}

// It's clear you'll need to execute the hook script after body it's loaded
ExecuteOrDelayUntilBodyLoaded(function () {
    hookScript();
});

Another trick it's to use Srch to override other functions. For example, a typical customer request is the change of the icon in SharePoint results.
Here you can work every display templates or do something more generic:

jQuery(document).ready(function () {

    if (typeof Srch !== "undefined" && typeof Srch.U !== "undefined") {
        Srch.U.getIconUrlByFileExtension = function(ctxCurrentItem) {
            return CustomFunction(ctxCurrentItem);
        }
    }
});

Enjoy ;)

Read More

Field validation formulas... and folder creation via CSOM

Lately i'm working on sandbox solutions, it seems customer prefers OOB solutions.
Indeed i created (inspired from some blog posts written by people who knows things for real) those formulas for phone number and email:

• Phone (+39 080-1234567):
=AND(
(FIND("+";[Phone];1)=1);
(ISNUMBER(VALUE(MID([Phone];2;FIND(" ";[Phone];1)-2))));
(ISNUMBER(VALUE(MID([Phone];FIND(" ";[Phone];1);FIND("-";[Phone];1)-FIND(" ";[Phone];1)))));
(ISNUMBER(VALUE(MID([Phone];FIND("-";[Phone];1)+1;LEN([Phone]))))))

• Email formula:
=AND(NOT(ISERROR(FIND("@";[Email])));NOT(ISERROR(FIND(".";[Email])));ISERROR(FIND(" ";[Email]));NOT(ISERROR(FIND(".";[Email];FIND("@";[Email];1)))))

SharePoint validation does not provide regular expression support (doh!) so you need to have fun with excel formula. The pro is that you can unit test formulas with excel, the con is that you have less power than regex.
But this post deals with another problem: when you set validation on a document library and you try to create a folder via CSOM on it (maybe also with SSOM but i haven't tested it) you have a very fun error: "List data validation failed."
So what to do? The right code is the following one, indeed forcing the ContentTypeId during folder generation

// Get content type instance ID
ContentTypeCollection listContentTypes = list.ContentTypes;
context.Load(listContentTypes, types => types.Include
     (type => type.Id, type => type.Name,
     type => type.Parent));
var result = context.LoadQuery(listContentTypes.Where
  (c => c.Name == "Folder" || c.Name == "Cartella"));
context.ExecuteQuery();
ContentType folderContentType = result.FirstOrDefault();

// Get author ID
FieldUserValue userValue = new FieldUserValue();
userValue.LookupId = owner.Id;

// Creating folder
ListItemCreationInformation newItemInfo = new ListItemCreationInformation();
newItemInfo.UnderlyingObjectType = FileSystemObjectType.Folder;
newItemInfo.LeafName = folderName;
ListItem newListItem = list.AddItem(newItemInfo);

newListItem["ContentTypeId"] = folderContentType.Id.ToString();
newListItem["Author"] = userValue;
newListItem["Title"] = folderName;
newListItem.Update();
context.Load(list);
context.ExecuteQuery();

Are you saying "Eureka"? Mmm now try to break role inheritance on this folder or try to make whatever update on the ListItem associated to this folder and... BAM... "List data validation failed" once again.

newListItem.BreakRoleInheritance(false, true);
RoleDefinitionBindingCollection role = new RoleDefinitionBindingCollection(context);
role.Add(context.Web.RoleDefinitions.GetByType(RoleType.Reader));
newListItem.RoleAssignments.Add(owner, role);
newListItem.Update();
context.ExecuteQuery();

The funny part is that the role inheritance was broken and the old roles cleared so, sadly, i've solved this managing the validation exception when i work on folder... not so elegant but it is what it is.
Have fun.

Read More

AngularJS to SP2013 REST repository

Hi there, this is just a fast post about repository in AngularJS using REST API on SharePoint 2013. Someone could say "why don't you use SP client context etc. etc." but this is not a pros/cons post, just a simple snippet in order to copy/paste the code when you'll need it :) .
I decided to post it since it was not so immediate and i had to play with verbs and item properties and maybe could help someone else (i also wrote that 2 months ago, so i don't remember details, so please don't ask me).
Ah, i haven't write method for delete since i don't need it, so have fun :p

repositories.service('ListRepository', ['$http', '$resource', function ($http, $resource) {

    var self = this;

    var listName = "List Name";
    var itemType = "SP.Data.List_x0020_NameListItem";
    var wsAddress = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getByTitle('" + listName + "')/";

    self.getListItem = function (key, onSuccess, onError) {
        var requestUrl = wsAddress + "items?$filter=Title%20eq%20'" + key + "'";

        $http({
            method: 'GET',
            url: requestUrl,
            headers: { "Accept": "application/json;odata=verbose" }
        })
            .success(function (response) { onSuccess(response); })
            .error(function (response) { onError(response); });
    };


    self.createListItem = function (item, onSuccess, onError) {
        var requestUrl = wsAddress + "items";

        item["__metadata"] = { "type": itemType };
            
        var stringifiedItem = JSON.stringify(item);
        var digest = $("#__REQUESTDIGEST").val();

        $http({
            url: requestUrl,
            method: "POST",
            headers: {
                "accept": "application/json;odata=verbose",
                "X-RequestDigest": digest,
                "content-Type": "application/json;odata=verbose"
            },
            data: stringifiedItem
        })
            .success(function (response) { onSuccess(response); })
            .error(function (response) { onError(response); });
        };


    self.updateListItem = function (item, onSuccess, onError) {

        self.getListItem(item.ExportToNasSetting_Source,
            function (response) {
                var oldItem = response.d.results[0];
                var requestUrl = wsAddress + "items(" + oldItem.Id + ")";

                item["__metadata"] = { "type": itemType };
                var stringifiedItem = JSON.stringify(item);

                var digest = $("#__REQUESTDIGEST").val();

                $http({
                    url: oldItem.__metadata.uri,
                    method: "POST",
                    headers: {
                        "accept": "application/json;odata=verbose",
                        "X-RequestDigest": digest,
                        "content-Type": "application/json;odata=verbose",
                        "X-Http-Method": "MERGE",
                        "If-Match": oldItem.__metadata.etag 
                    },
                    data: item
                })
            .success(function (response) { onSuccess(response); })
            .error(function (response) { onError(response); });
            },
            onError);        
    };

}]);

Please note  that this repository could be considered quite generic, you need to change variables listName and itemType with the ones you use.
Also the method getListItem, use filter query string to filter by title, feel free to play with this method as you like.

Read More

Embarassing stories: console.log with IE Enterprise mode

I know, you're here and your soul is yelling: "why in the world, even if i'm working on IE11, there is someone who uses Enterprise mode, so i have to work in IE8 emulation mode?"
I don't know why, but this is something which makes me hate this work, even because right now i'm working a lot with AngularJS.
This post explains some simple tricks to make AngularJS and IE8 happy together: AngularJS and IE8 - Happy Together? and helped me a lot.
In my case i had also a custom action which was not working with Enterprise Mode. After banging my head for some hours i noticed that the code worked only when i use developer tools (you know, the wonderful F12 key), otherwise no joys for me.
The problem was simple: IE8 and console.log are not very happy together.
So i had to declare the custom action like you can see below:
You can see on top consoleLogHelper.js, this script, if console.log is undefined, instantiate it so the code does not explode.

Read More

Obsolete

As in the subject, i cannot work no more on ASPFM and VLibs, since:

• ASPFM could be replaced with the new wonderful Azure farm provisioning
• VLibs simply requires time, and right now, in my free time, i'm working on something more B2C

That's it :) .

Read More

Howto search for memory leak and save your marriage

Someone said (on friday afternoon): "your application got a memory leak and it's your fault".
Because of my wife became very nervous when i gotta work during the weekend i asked some help in order to understand how to work this problem as fast as possible, so let's start with credits to Iron Man and Mr. Pumpkin (only nicknames here: privacy is privacy).
This is what i learned from pro-people which helped me to do a drill-down analysis.
Pre-conditions:

1. You know farm sometimes goes down
2. You must have front end IIS logs
3. You know there are lots of objects in memory (as i can see in w3wp.dmp)
4. You need ULS logs

The bug-hunting process deals with:

1. Find the use case that reproduce the memory issue
2. Replicate the case in your dev envinronment
3. Analyze the issue
4. Conclusions

Step 1 - Find the use case that reproduce the memory issue
To accomplish the first task, i have taken the IIS logs in order to find out "who was doing what" on the web application. IIS logs are too long to use a notepad, so Mr. Pumpkin said: "Use lizard, download it from here http://www.lizard-labs.com/log_parser_lizard.aspx"
Lizard helps you to make query with SQL on log files, IIS logs contain lots of rows with those informations:

• date
• time
• s-sitename
• s-computername
• s-ip
• cs-method
• cs-uri-stem 
• cs-uri-query 
• s-port 
• cs-username 
• c-ip cs-version 
• cs(User-Agent) 
• cs(Cookie) 
• cs(Referer) 
• cs-host 
• sc-status 
• sc-substatus 
• sc-win32-status 
• sc-bytes 
• cs-bytes 
• time-taken

etc But as Mr. Pumpkin said, please be aware IIS log are in UTC, so take care of using this query (maybe restricting where conditions on a particular time-slot near to the memory issue)

SELECT
   TO_LOCALTIME(TO_TIMESTAMP(date, time))
   ,date
   ,time
   ,s-sitename
   ,s-computername
   ,s-ip
   ,cs-method
   ,cs-uri-stem
   ,cs-uri-query
   ,s-port
   ,cs-username
   ,c-ip
   ,cs-version
   ,cs(User-Agent)
   ,cs(Cookie)
   ,cs(Referer)
   ,cs-host
   ,sc-status
   ,sc-substatus
   ,sc-win32-status
   ,sc-bytes
   ,cs-bytes
   ,time-taken
FROM 'C:\Users\Varro\Downloads\iis_frontend47.log'

in this way, column TO_LOCALTIME(TO_TIMESTAMP(date, time)) will give you the localtime. Now it's search time, filter on whatever you want. In my case, i take a look around sc-status (request status code) and time-taken in order to find out the most time consuming call, and i found some interesting data which helped me to replicate the behaviour users had before memory allocation went outta control.

Step 2 - Replicate the case in your dev envinronment
This is one of the most interesting part of the job, now i have the users behaviour i did the same actions on my system, attaching the memory profiler on Visual Studio, below the steps Iron Man explained to me:

1. Create a new performance session

 

2.  Change properties of performance session and choose performance counters

• NET CLR Memory/# Bytes in all Heaps
• .NET CLR Memory/Large Object Heap Size
• .NET CLR Memory/Gen 2 heap size
• .NET CLR Memory/Gen 1 heap size
• .NET CLR Memory/Gen 0 heap size
• .NET CLR Memory/Induced GC
• Process/Private Bytes
• Process/Virtual Bytes

3. Attach performance session  to SP process

4. Execute use case and monitor task manager, when you see memory going up, create some memory dumps of the process

5. When you ends the use case, stop everything and watch results ;)

Pay attention to this, you could need to run this command from che Visual Studio command prompt in order to make the profiler work

   vsperfclrenv /globalsampleon
   iisreset

Step 3 - Analyze the issue
What i discovered is that, effectively, there was a high memory consumption, so i tried to discover who could be the guilty method.
I can do this using memory profiler report and debug2diag (https://www.microsoft.com/en-us/download/details.aspx?id=49924) in order to process DMP files.

Memory DMP report

Memory profiler report

Iron Man also explained that i need to replicate the same pattern of memory allocation (as i could see on the customer w3wp.dmp) in order to have a smoking gun. So, using also the debugger to freeze the code in some interesting points in order to make some specific memory dump from task mager.

Step 4 - Conclusions
This pretty complex analysis helped me out to discover our application is quite slow in some particular circumstances, but even if some operations stressed so much memory allocation and garbage collector (some method like SubMethod1 could be more efficient), it's also true memory is released after those operation, so, if farm fall down when users do this operation... maybe it's time to give your farm just a little bit RAM.

That's all folks.

Read More