How to get started with Office 365 API Development

Hi All,

I have been asked by clients: How to get started with Office 365 API Development ?

They want to integrate office365 capabilities in their existing systems and applications.

In this blog i am providing links on how to get started with authentication, accessing mail, calendar, contacts and providing an amazing *FREE* training course from MS MVA (Microsoft Virtual Academy) site:

1) Office 365 API platform overview:

https://msdn.microsoft.com/en-us/office/office365/howto/platform-development-overview?f=255&MSPPError=-2147217396

2) Samples to integrate with Office 365, specially what you mentioned mail, calendar...etc.:

Check out under Web, It has MVC Web samples to integrate calendar, contacts..etc

https://msdn.microsoft.com/en-us/office/office365/howto/starter-projects-and-code-samples

3) Office 365 app authentication concepts, this will provide different scenarios to authenticate users against o365 (Important):

https://msdn.microsoft.com/office/office365/HowTo/common-app-authentication-tasks

4) Great FREE training course for office 365 development training:

http://www.microsoftvirtualacademy.com/training-courses/introduction-to-office-365-development?m=10072&ct=31602

5) Calendar, Mail and Contact REST API reference:

https://msdn.microsoft.com/office/office365/APi/mail-rest-operations

https://msdn.microsoft.com/office/office365/APi/calendar-rest-operations

https://msdn.microsoft.com/office/office365/APi/contacts-rest-operations

Hope this helps, and feel free to send any questions over to help.

Error when running web applications that reference SharePoint dlls in IIS Express

Hi All,

I was developing a web application that reference SharePoint 2013 dlls, and since SharePoint server dlls in general runs on a 64 bit environment, i want to use IIS express in Visual Studio 2013 without the need to use full local IIS that runs on 64 bit.

I was getting this error message when i am running my web application:

Could not load file or assembly or one of its dependencies. 

and i was seeing in the yellow error page that the VS 2013 is using IIS express under c:\ program files (86)\ IIS Express folder which is the 32 bit.


I want to switch IIS express to 64 bit so i will be able to run my web application that references 64 bit SharePoint dlls. here is the solution for this:

From Visual Studio 2013 IDE:

Click on :  Tools --> Options --> Projects and Solutions --> Web Projects  --> Use the 64 bit version of IIS Express --> check this checkbox and save.


Once you save, make sure you exit any instance of the IIS express and try to re-run your project! It will run with no issues!

Enjoy!

Developer Dashboard doesn't show for anonymous users in SharePoint 2013

Hi All,

I was working on a public SharePoint site and i wanted to enabled the developer dashboard utility. I Ran the following powershell command to enable the developer dashboard:

Turn On Developer Dashboard Powershell Command:

$content = ([Microsoft.SharePoint.Administration.SPWebService]::ContentService)
$appsetting =$content.DeveloperDashboardSettings
$appsetting.DisplayLevel = [Microsoft.SharePoint.Administration.SPDeveloperDashboardLevel]::On
$appsetting.Update() 


**If you want to turn it off, just change the On flag to Off and you are good.

After that, I was able to see the developer dashboard icon in the Central Administration site but not on my public SharePoint site.

So basically, I found that you have to grant permissions for anonymous users to access this utility since it is not accessible for anonymous users even if the utility is enabled.

How to enable Developer Dashboard for anonymous users:

$contentService = [Microsoft.SharePoint.Administration.SPWebService]::ContentService
$dashboardSettings = $contentService.DeveloperDashboardSettings
$dashboardSettings.RequiredPermissions = [Microsoft.SharePoint.SPBasePermissions]::EmptyMask
$dashboardSettings.Update()


By default, the required permissions property for the developer dashboard utility is "AddAndCustomizePages" which is available for authenticated users who have this permission, so you need to changed it to no permissions by setting the value to EmptyMask which means in SharePoint permissions world no permissions are needed.

Note: This is a good technique while you are developing, testing and in the user acceptance test phase of any SharePoint implementation project. when you don't need this utility it is strongly recommended to turn it off on production environment.

Hope this helps!


References:
1) SharePoint base permissions:
http://msdn.microsoft.com/EN-US/library/ms412690

The memory usage on the current server is about {0}. This can cause the eviction or throttling of the Distributed Cache Service

Hi,

After enabling the usage and data collection service in SharePoint 2013, i was getting the following warning in windows event log:

The SharePoint Health Analyzer detected a condition requiring your attention.  The Current server is running low on memory.
The memory usage on the current server is about {0}. This can cause the eviction or throttling of the Distributed Cache Service/
Check the memory usage on the machine.And try to free up some memory or add more ram on the machine. For more information about this rule, see "http://go.microsoft.com/fwlink/?LinkID=224261".

How to tackle this memory consumption and performance issue in SharePoint 2013:

First, if in you are not utilizing the document activities, feeds or micro-blogging services in SharePoint so it is recommended to stop the Distributed Cache Service. this will give you a great performance boost in your box.

To get an understanding of the benefits of Distributed Cache Service in SharePoint read this TechNet post- Section "Benefits of Distributed Cache":

http://technet.microsoft.com/library/jj219700(office.15).aspx

Second, you can minimize the number of days you want to store SharePoint logs on the disk to free up some space. here is the steps:

1) Open Central Administration,

2) Click on Monitoring.

3) Under Reporting, Click on Configure diagnostic logging.

4) Scroll down to Trace log, where you can change the number of days to store log files from 14 to 5 days or you can specify the size of the log files in GB which gives you control over the size of these growing files.

Hope this helps.

RegisterSod is undefined

Hi folks,

I ran into an issue when i was working on a SharePoint hosted app, when the application loads it keeps throwing errors in the IE development tool that "RegisterSod is undefined".

I was deploying the application to the following site url:

http://SP2013/sites/appCatalog

The cause of this problem is that i didn't have a site in the root site collection, so i opened the Central Administration and created a site collection for the same web application that i am using to deploy my SharePoint apps to "appCatalog" site.

Re-deployed my application! it works!

Hope this helps!

-- ME

SharePoint Hosted App keeps prompting the username and password in SharePoint 2013

Hi All,

I was deploying a SharePoint Hosted App on my on-premise SharePoint 2013 Development VM and i was getting the Windows login window and it keeps prompting the username and password, then after 3 times the page goes blank with no errors or content in the default page.

After searching and trying different things, the fix wasn't not just to disable the loopback check in the registry that i had already as a DWORD 64 bit entry that is required to Crawl external SharePoint public websites.

The fix was to delete the DWORD 64 bit and replace it with 32 bit and the application starts instantiate with no issues and it starts working!.

Hope this tip helps.

Enjoy SharePointing!!

How to get the url for a SharePoint EndPoint hosted in a Service Application

Hi All,

This blog post is for SharePoint exposed EndPoints through Service Applications. If you are developing or deploying a custom Service Application in SharePoint 2010/2013 and you are exposing a WCF EndPoint Service EndPoint through the service application this blog post is helping you how to identify and get the Url for the exposed WCF EndPoint.

You need to test and make sure that the service application is working properly before checking any client application or webpart code that is consuming or integrating with this endpoint. Specially, if your endpoint might be having issues related to assembly dependencies or missing files that causes the service application to throw exceptions when it is being called from the client applications.

When you develop a service application in VS 2012/2013, the service application creates an application pool in the IIS, since you give it a name when you install it, and this expose a service instance where you can access it through a url!

To develop Service application for SharePoint 2010/2013, I strongly recommend this starter kit since it gives you a lot of ground work is done for you and you just need to code your logic in it.

http://chocolatey.org/packages/sastarterkitvs2012

Great shout out to Adam Toth !

So the question that needs an answer is:

What is the Url for the exposed WCF end point service that is hosted in a SharePoint Service Application?

1) You need to know the Service Application GUID that is installed in your farm, to do this Open SharePoint PowerShell and write below cmdlet:

Get-SPServiceApplication | select id,name

This will list all deployed service applications and provides the Service Application GUIDs.

2) Open IIS Manager, Expand SharePoint Web Services.
3) Look for the GUID that is associated with your service application.
4) Select the your endpoint and click on the browse from the IIS manager right pane.
5) Add the endpoint name "WcfEndPoint.svc" to the url !

http://localhost:32843/c20d8cbe444f4334a212ccba687dfbd7/MYWCFEndPoint.svc

Enjoy!

Web Part or Web Form Control on this Page cannot be displayed or imported. while adding a custom webpart in SharePoint 2013

Hi All,

I was developing a custom SharePoint web part for SharePoint 2013 using VS 2013, after i deployed the wsp and when i was adding the web part, i was getting the following error message:

Web Part or Web Form Control on this Page cannot be displayed or imported. The type could not be found or it is not registered as safe.

This is a common SharePoint error when you try to load custom user controls (or Visual Web Parts) into a SharePoint page. 

Any custom control has to be added as a safe control in SharePoint, before VS 2013 we used to add entries in the site web.config or code to add the safe control either programmatically to the site. but you do not need to do this in Visual Studio 2013 since it has a tool to add safe controls through a UI by specifying the namespace and the type and your control will be loaded with no issues or errors.

In VS 2013, follow these steps:

1) Double click on the package item.

2) Under Advanced tab.

3) If the custom control exists in an assembly so when you add the assembly that contains your user control click on add safe control button and specify your assembly & type for their user control.

Or even if the custom control is contained in a project within your solution, so when you add the project output in the package explorer UI, you can still add the user control as a safe control by specifying the namespace and type fields.

4) Safe the changes, build and deploy.

5) Refresh the SharePoint page.

6) Add your custom web part and you will not get the error message.

Enjoy!

Transform XML using XSLT document to HTML output in IE

Hi All,

I was developing a JavaScript function that takes an xml document for a HTTP response and i want to transform it to HTML in my page or web part (in SharePoint world).

I had code that uses transformNode that is not supported by IE anymore starting from version 9 and i am using IE 11.

Actually, i spend a quite number of hours to find out what is the best way to load an XML document and transform it using a XSLT file into a HTML output in IE 11 since i was getting different errors.

You will find a lot of posts/blogs are referring to this code snippet to transform xml using xslt file in IE:

var xml = new ActiveXObject("Microsoft.XMLDOM"); 
var xslt = new ActiveXObject("MSXML2.FreeThreadedDOMDocument");
xml.load("data.xml");
xslt.load("data.xls");

var processor   = new ActiveXObject("Msxml2.XSLTemplate");
processor.stylesheet = xslt;

var objXSLTProc = processor.createProcessor();
objXSLTProc.input = xml;
objXSLTProc.transform();
var output  = objXSLTProc.output;

Above code doesn't work in IE 11, for 2 reasons:
1) IE 11 doesn't support load method! you have to use loadXML instead.
2) you have to get the string representations of both xml and xslt to be able to use loadXML and then process the output.

Here is the code snippet that works with IE 11:

function transformXml(xml)
{

var loadedXslt = LoadXSLTDocumentInIE("myxslt.xslt");

  // Load the XML Document
  var xmlDoc = new ActiveXObject("Msxml2.DOMDocument.3.0");
  xmlDoc.async = false;
  xmlDoc.resolveExternals = false;
  xmlDoc.loadXML((new XMLSerializer()).serializeToString(xml));
   console.log(xmlDoc.parseError.reason); // for debugging, to make sure there is no errors after loading the document.

    // Load the XSL file
    var xslt = new ActiveXObject("Msxml2.XSLTemplate");
    var xslDoc = new ActiveXObject("Msxml2.FreeThreadedDOMDocument");
    xslDoc.async = false;
    xslDoc.loadXML(loadedXslt.responseText);
    xslt.stylesheet = xslDoc;

    var xslProc = xslt.createProcessor();
    xslProc.input = xmlDoc;
    xslProc.transform();
    return xslProc.output;

}

// function to load XSLT document by name, this document has to be accessible through the web browser.
function loadXSLTDocumentInIE(fileName) {
   xhttp = new ActiveXObject("Microsoft.XMLHTTP");
   try {
        xhttp.responseType = "msxml-document";
    } catch (e) {
        //console.log("couldn't set the response type msxml in IE");
    }
    xhttp.open("GET", fileName, false);
    xhttp.send("");
    return xhttp;
}


This code works like a champ in IE 11 :)

Hope this helps.

How to get the Query String value in SharePoint 2013 Search Results page

Hi All,

I was working to customize the search results web part by customizing the Search Result Display Item template and the Item Hover Panel as well.
If you want to read more about how to customize the search display templates read below blog post:
http://moustafa-arafa.blogspot.com/2014/01/customize-sharepoint-2013-search.html

I was trying to get a value of the query string that SharePoint 2013 passes once you hit enter in the Search Box, and since this control doesn't have an ID, I decided to pull the value from the query string that has a key "K".

After i developed this scenario, I found that SharePoint if you ran another search keyword for the second time, It would append the newer keyword with anchor (#) and will keep the old search keyword in the query string with key "K"

For example, When you run the search for the first time with keyword "copyright":

http://myPortal2013/sites/sp2013/search/pages/results.aspx?k=copyright

If you changed the search keyword to "digital", here is the new url would be:

http://myPortal2013/sites/sp2013/search/pages/results.aspx?k=copyright#digital

In my case, i want to get the "digital" and not "copyright" keyword so i want to parse the Url and get the latest search keyword. Here is the code to accomplish this:

This code i wrote it in the outer div element of the body of your display template html file.

Note: You can use this variable "sParam" as you like in the Display Template html file.

Hope this helps.

-- ME

Customize SharePoint 2013 Search Display Templates

Hi Folks,

In this post i will show how to customize SharePoint 2013 Display Templates and specifically the Search Templates.

In SharePoint 2013, you can customize the search results web part by editing the configuration of the web part. but if you want to customize the rendered output/layout in this case you need to use SharePoint Designer 2013 or any HTML editor to customize the search result item or the hover panel.

How to customize the search result item in Search Results web part:

1) From Site Settings page, Click on Master Pages and Page Layouts.
2) Click on Display Templates, This folder contains all SharePoint 2013 Display Templates.
3) You will find wide range of folders under Display Templates, Since we need to customize the search display template, click on Search Folder.
4) You need to find 2 items in this list, here are these files we need to download a copy of them:
     Item_Default.html
     Item_Default_HoverPanel.html
5) Select these 2 files and download a copy of them. DO NOT SELECT THE .JS files with the same name. We do not need them, they will be created for us!

6) Once you select both files, click on any little arrow beside the file title and click on Download a copy.
7) Copy these file and rename them so this will be your customized template to be associated with the search results web parts.

For example: let us call them: Item_Default_Custom.html , Item_Default_HoverPanel_Custom.html

7) Upload both files to the list, you will notice a .js file is created for each of the files.
8) Now, we have our own custom template that we can customize in SharePoint.
9) Open SharePoint Designer 2013, From All Files --> _ catalogs  --> masterpage --> Display Templates --> Search
10) You will find both files we just uploaded, here is it is up to you. If you want to customize the search result item, then we will open Item_Default_Custom.html.
If we want to customize the Hover panel when you hover on a search result item, then we have to open Item_Default_HoverPanel_Custom.html.

Tip: In all cases, you should not modify the .js version of the html files. The Javascript file is a SharePoint file that is being updated/generated every time you save any change in the html files.

The changes in the search html template, it requires standard web development experience for editing html and writing javascript code.

Below is a very useful blog, It shows some techniques in how to customize the html files and embed some javascript code If you are not familiar with SharePoint/Web development.

http://sharepointfieldnotes.blogspot.com/2012/11/customizing-sharepoint-2013-search.html


How to configure Search Results Web Part to load your custom Display Template:

Once you customize the search display template, One last step is to configure the Search Results web part to load your template, Below is how to do it:

Your custom Search Display Result item will be loaded in the Display Templates in the web part configuration and once you select it, any change you do in the html will be reflected once you refresh the page in SharePoint 2013.

Hope this helps.

Error when creating search center site in SharePoint 2013

Hi folks,

I was trying to create an enterprise search center site in my SharePoint 2013 VM. When i was clicked on Create button i was getting the well known yellow error page in SharePoint.

I straight up checked the logs and i found the following error message:

Exception in EnsureFeatureActivatedAtWeb: Microsoft.SharePoint.SPException: The SharePoint Publishing Infrastructure feature must be activated at the site collection level before the publishing feature can be activated.

So,  I went to the site settings  --> site collection features --> then i activated the SharePoint Publishing infrastructure feature.

Then, I went back to create a subsite --> selected enterprise search center. I entered all required information and when i clicked on create. the site is created with no errors!

Platform Not supported Exception when running Visual Studio Web Application

Hi All,

While i was developing a web application that creates an instance of a custom service application using Visual Studio 2013, I was getting this error:

Exception Type: PlatformNotSupportedException

Details:

Microsoft SharePoint is not supported in 32-bit process. Please verify that you are running in a 64-bit executable.

I have another SharePoint web part project that connects with no issues to the SharePoint custom service app, but this web application is throwing this exception! so, the error is related to the IIS express which is the default IIS for VS 2013 web projects.

The error will exist or be thrown when you create any SharePoint object such as: SPSite to connect and then implement your logic.

I googled little bit and i found that some blogs were saying to un check the prefer 23 bit under Build tab in the project properties, In my case this check is grayed out so i don't have the option to un check it.

In addition to that, by default all SharePoint projects or Web Projects are targeting AnyCPU, so you shouldn't change this specially if you are connecting or integrating with SharePoint.


The fix:
1) Right click on the web project that you are having this error and click on the web tab.
2) Under Servers, change the selection from IIS express to Local IIS, This will use the local IIS instance which is a 64 bit installation of your SharePoint Development machine.
- Note: This will allow VS 2013 to create a virtual directory and attach the process, so you will be able to debug once you hit F5.
3) Run you project, the code works with no exception when creating any SharePoint object such as a Site or Web...etc.

SharePoint Development Machine Configuration: Win Server 2008 R2 64 bit, VS 2013 latest release.

Happy SharePointg!!

Sorry, Something went wrong when deploying WCF service in SharePoint 2013

Hi,

While i was developing a RESTful WCF Service that is targeting SharePoint 2013 using VS 2013, I was getting the following "White" error page when i access the WCF service after deploying my project in VS 2013.

As shows in the above screen shot, The error is related to getting and loading the right assembly for my WCF service class type, I know the GUID in my WCF class is matching the GUID in the svc file. But what is the problem ?

I found out that the solution for this problem is SharePoint doesn't accept the upper case letters in the GUID.

This was my GUID:  CDCC536B-37BB-4CC3-BF20-263AB66926D5

I changed the GUID by replacing the upper case characters to lower case characters in the Service class and in the SVC file, then save and deploy! It works!

Happy SharePointing!

Where is Microsoft.SharePoint.ApplicationPages.Administration.dll in SharePoint 2013

Hi,

If you are developing custom application pages in SharePoint Central Administration, you will be in need to add this reference in your project, Here is the path for this dll:

c:\program files\common files\microsoft shared\web server extensions\15\CONFIG\ADMINBIN

Enjoy!

Tips on SharePoint 2013 Branding and UX Design

Hi All,

I had the opportunity to work with SharePoint 2013 Branding and building new UX related components such as: Master Pages, Page Layouts and Content Types development on a SharePoint 365 site or SharePoint 2013.

I found some interesting tips and tricks and i'd like to combine them all in this blog post for End Users, Designers, Architects and anyone who works in SP branding and customization.

Here are some general information and tips to know ahead before getting in creating new UX SharePoint components:

1) If you are new to SharePoint and you want to understand how MasterPages are connected to page layouts and actual pages, I recommend reading this article first:
http://msdn.microsoft.com/en-us/library/jj191506.aspx

2) In SharePoint 2013, You can create your master page in any html editor --> sweet!
3) Once you are signed off on the new look and feel of the new master page in HTML, A good option: you can map the new master page in html with all related javascript and CSS files to the SharePoint Master page gallery, Here is how to do it:
http://msdn.microsoft.com/en-us/library/jj733519.aspx

4) Another option is: you can directly convert the html master page to an actual master page; where SharePoint will create a new file with .master extension and add SharePoint attributes and place holders used by SharePoint, here are the steps using the design manager:
http://msdn.microsoft.com/en-us/library/jj822370.aspx

5) Design Manager is a new feature in the publishing sites, you can use it to brand public facing SharePoint sites and Office 365 sites. Read an overview about How to use Design Manager in SharePoint 2013:
http://msdn.microsoft.com/en-us/library/jj822363.aspx

6) Once you create your new master page, you might need to create new set of page layouts that provide nicer UX to your visitors, This article shows how to create new page layouts in SharePoint 2013 and office 365:
http://msdn.microsoft.com/en-us/library/jj822368.aspx

7) If you want to have different UX for Mobile and tablet users, you should create different channels aka different master pages and page layouts for different targeted devices, this article describes how to implement this in your site:
http://msdn.microsoft.com/en-us/library/jj862343.aspx


Hope this helps, drop me a line if you have any questions or clarifications.



Reference:
- Build Sites for SharePoint 2013:
http://msdn.microsoft.com/en-us/library/jj163242.aspx

SharePoint 2013 Strategy client object model (CSOM) tips

Hi Folks,


I have consolidated some useful tips about CSOM in SharePoint 2013 from training vidoes i found on MSDN and i decided to put it on a blog post so it is easier and faster to get your notes out of them.

1)      In SharePoint 2010, you can’t access client.svc to conduct client calls to SharePoint Object model, instead you use .NET, Silverlight, or JavaScript client library to conduct client call to the WCF client.svc on the server from your client applications.

2)      In SharePoint 2013, you can directly access client.svc from REST clients.

3)      In SharePoint 2013, Client.svc accepts HTTP verbs such as: GET, POST , PUT.

4)      Extended CSOM API that supports more functionality such as: Search, Taxonomy, Publishing, Analytics, Workflow and E-Discovery.

5)      “_api” is a new alias for “_vti_bin/client.svc” to access the end point.

6)      CSOM code in SharePoint 2010 should be migrated to SharePoint 2013 CSOM with no issues.

7)      ListData.svc still available for backward compatibility applications from SharePoint 2010 into SharePoint 2013.

8)      It is not recommended to use ListData.svc in your SharePoint 2013 CSOM code, Use OData API instead, check #12.

9)      You can use fiddler to monitor client queries in CSOM code, especially when you call ExecuteQuery() in CSOM code.

10)   Using IE developer tool and Fiddler are essential when you code using JavaScript to call sp.js in CSOM. It also allows seeing the details of the request and JSON response content.

11)   FormDigest is being created the first time when calling ExecuteQuery for the first time; this is being used on all consecutive calls.

12)   Simply OData URIs as follows to access the web:

http://www.contoso.com/_vti_bin/client.svc/web

Simplified: http://www.contoso.com/_api/web

To access “Announcements” list:

http://www.contoso.com/_api/web/getByTitle(‘Announcements’)

Hope this notes would help.

Reference:

SharePoint 2013 CSOM and OData API Training Videos

http://msdn.microsoft.com/en-us/sharepoint/fp123633.aspx

 

What’s new in Records Management and Compliance in SharePoint 2013

Hi Folks,

I’m writing this article about what’s new in Records Management and Compliance in SharePoint 2013. SharePoint 2013 has extended the retention policies to be applied on the site level!.

Having the site retention policies on the site level is a cool feature, especially for most of the small/medium organizations that creates sites for their departments and not site collections.

Another cool feature which is the site policy also applies on SharePoint 2013 Foundation, so you can apply your site retention polices if you have SP 2012 Foundation instance.

Site Level Retention:

You can manage retention policies to SP sites and exchange 2013 team mailboxes that are associated with the sites.

Compliance officers create policies, which define the following:

n  The retention policy for the site and the associated exchange team mailbox, if any is associated.

n  What causes the project to be closed?

n  When a project should expire.

Here are the steps to apply compliance and retention policies:

1)      The site owner creates a SharePoint site.

2)      The site owner creates an exchange server 2013 team mailbox.

3)      The site owner assigns/selects the appropriate retention policy template.

4)      The site owner invites team members to join the site.

5)      All emails and documents among the site team members, the selected policy will be applied.

6)      When the project is completed, the site owner closes the project.

7)      Once the project is closed by the site owner, all project folders in outlook 2013 User Interface will be removed.

8)      When the project expires as per the selected policy, all associated artifacts to this project will be deleted.

 Hope this helps.

Useful Urls:

** Overview of site permissions in SharePoint 2013

http://technet.microsoft.com/en-us/library/jj219569.aspx

 

 

New BCS Enhancements in on-premise and online SharePoint 2013

Hi Folks,

I'm writing this blog to list the new BCS features and enhancements in SharePoint 2013. SharePoint 2013 is released with great features for developers and IT Admins in terms of ease of use connecting to external sources and the ability to develop self-contained SharePoint applications that connect to external sources. In this article i will describe the new features and enhancements when building solutions on top of SharePoint 2013.

BCS enhancements in on-premise SharePoint 2013:

1) Automatic generation for BDC Models for OData data sources:
In SharePoint 2013, you are able to connect to OData endpoints and generate BDC models. You must use Visual Studio 2010 to create BDC models for OData data sources.
Then you can import these models in BDC Catalog on your farm-scoed external content types or added on a SharePoint App. VS 2010 will generate  BCS operations for all OData Operations (Get, Put, Post and Delete).

2) The ability to create SharePoint App-Scoed External Content Type:
When creating BDC models in VS 2010, you will be able to add App-Scoped external content types that will be used only on the associated SharePoint App.

3) Performance improvements in external lists:
SharePoint 2013 has introduced new enhancements and improvements to minimize the load on the database servers in the SharePoint farm and by increasing the speed of list rendering.
The ability to perform paging, filtering, and sorting of the external list data before it is sent to SharePoint is a great feature.

4) Limiting the records returned by external system:
In the BDC model, users can specify the number of records in the list that they want displayer per page.

5) Data Source Filtering, Sorting and export to excel:
In SharePoint 2013 you are able to filter and sort the data exposed through the an external list. So, in this case the external system filter and sort the data before send it back to the external list.
Also, you can export an external list to excel and in this case you get all list content and structure it shows in the browser.

BCS enhancements in SharePoint Online:

1) REST (CSOM): REST Api is available for web and mobile devlopers to consume SharePoint data using Client Obect Model for the REST Api.

2) BCS Client runtime supports side by side Office 2010 and Office 2013 installaion.

Hope this helps.

-Mostafa Elzoghbi

Understanding Crawl and Index components interactions in SharePoint 2013 - Part 1

Hi,

I’m writing this post to describe the main components in SharePoint 2013 Search and related components.

As you know, SharePoint 2013 has a slight change in Search components than SharePoint 2010 since they re-architected search components and their dependencies from the previous version.

I will explain the purpose of the 4 main components (marked with asterisk below) so it will be easier to understand and help you when you plan for your SharePoint Farm deployment since Search is a crucial component to scale for SharePoint collaboration and user adoption.

Main SharePoint 2013 Search Components:

1)      Crawl component*.

2)      Content Processing component*.

3)      Analytics processing component.

4)      Index component*.

5)      Query Processing component*.

6)      Search Administration component.

*These components are covered in this article.

SharePoint 2013 Databases:

1)      Crawl database.

2)      Link database.

3)      Analytics reporting database.

4)      Search Administration database.

First and foremost, is to understand the Crawl components interaction, here I will talk about 3 main components:

a)      Crawl Component

b)      Content Processing component

c)       Crawl database

n  Crawl component: The crawl component is responsible for crawling content sources. It delivers crawled items – both the actual content as well as their associated metadata – to the content processing component.

The crawl component invokes connectors or protocol handlers that interact with content sources to retrieve data. Multiple crawl components can be deployed to crawl simultaneously.

Note: The crawl component uses one or more crawl databases to temporarily store information about crawled items and to track crawl history.

n  Crawl database: The crawl database contains detailed tracking and historical information about crawled items. This database holds information such as the last crawl time, the last crawl ID and the type of update during the last crawl.

n  Content processing component: The content processing component is placed between the crawl component and the index component. It processes crawled items and feeds these items to the index component.

The content processing component transforms crawled items into artifacts that can be included in the search index by carrying out operations such as document parsing and property mapping.

Both the content processing component and the query processing component perform linguistics processing. Examples of linguistics processing during content processing are language detection and entity extraction.· The content processing component writes information about links and URLs to the link database.

 Below shows the dependency data flow between Crawl and Index components:

Crawl Component àStore à Crawl DB àFetch/Process crawled items

à Content processing àcontent processing & extractionà Index Component

Second, once the data has been processed to Index Component, The next step contains other 3 main components are interacting to provide search capability in SharePoint:

a)      Index Component

b)      Index Partition

c)       Query Processing Component

 n  Index component: An index component is the logical representation of an index replica.

In the search architecture, you have to provision one index component for each index replica.

The index component receives processed items from the content processing component and writes those items to an index file.

The index component receives queries from the query processing component and provides results sets in return.

Queries are sent to the index replicas through the query processing component. The system routes and load balances the incoming queries to the index replicas.

 

n  Index partition: An index partition is a logical portion of the entire search index.

The search index is the aggregation of all index partitions.

n  Query processing component: The query processing component is between the search front-end and the index component.

The query processing component analyzes and processes search queries and results.

Both the query processing component and the content processing component perform linguistics processing. Examples of linguistics processing during query processing are word-breaking and stemming.

When the query processing component receives a query from the search front-end, it analyzes and processes the query to attempt to optimize precision, recall, and relevancy. The processed query is then submitted to the index component.

The index component returns a result set based on the processed query back to the query processing component, which in turn processes that result set before sending it back to the search front-end.

 

 

 

There are my thoughts on the main Search interacting components in SharePoint 2013, Drop me a line if you have any additions or questions.

 

 

- Reference:
http://technet.microsoft.com/en-us/library/jj219738.aspx

Hope this helps.

-ME