Monday, June 24, 2013

New BCS Enhancements in on-premise and online SharePoint 2013

Hi Folks,

I'm writing this blog to list the new BCS features and enhancements in SharePoint 2013. SharePoint 2013 is released with great features for developers and IT Admins in terms of ease of use connecting to external sources and the ability to develop self-contained SharePoint applications that connect to external sources. In this article i will describe the new features and enhancements when building solutions on top of SharePoint 2013.

BCS enhancements in on-premise SharePoint 2013:

1) Automatic generation for BDC Models for OData data sources:
In SharePoint 2013, you are able to connect to OData endpoints and generate BDC models. You must use Visual Studio 2010 to create BDC models for OData data sources.
Then you can import these models in BDC Catalog on your farm-scoed external content types or added on a SharePoint App. VS 2010 will generate  BCS operations for all OData Operations (Get, Put, Post and Delete).

2) The ability to create SharePoint App-Scoed External Content Type:
When creating BDC models in VS 2010, you will be able to add App-Scoped external content types that will be used only on the associated SharePoint App.

3) Performance improvements in external lists:
SharePoint 2013 has introduced new enhancements and improvements to minimize the load on the database servers in the SharePoint farm and by increasing the speed of list rendering.
The ability to perform paging, filtering, and sorting of the external list data before it is sent to SharePoint is a great feature.

4) Limiting the records returned by external system:
In the BDC model, users can specify the number of records in the list that they want displayer per page.

5) Data Source Filtering, Sorting and export to excel:
In SharePoint 2013 you are able to filter and sort the data exposed through the an external list. So, in this case the external system filter and sort the data before send it back to the external list.
Also, you can export an external list to excel and in this case you get all list content and structure it shows in the browser.

BCS enhancements in SharePoint Online:

1) REST (CSOM): REST Api is available for web and mobile devlopers to consume SharePoint data using Client Obect Model for the REST Api.

2) BCS Client runtime supports side by side Office 2010 and Office 2013 installaion.

Hope this helps.

-Mostafa Elzoghbi

Thursday, June 20, 2013

Understanding Crawl and Index components interactions in SharePoint 2013 - Part 1


I’m writing this post to describe the main components in SharePoint 2013 Search and related components.
As you know, SharePoint 2013 has a slight change in Search components than SharePoint 2010 since they re-architected search components and their dependencies from the previous version.
I will explain the purpose of the 4 main components (marked with asterisk below) so it will be easier to understand and help you when you plan for your SharePoint Farm deployment since Search is a crucial component to scale for SharePoint collaboration and user adoption.

Main SharePoint 2013 Search Components:

1)      Crawl component*.

2)      Content Processing component*.

3)      Analytics processing component.

4)      Index component*.

5)      Query Processing component*.

6)      Search Administration component.

*These components are covered in this article.

SharePoint 2013 Databases:

1)      Crawl database.

2)      Link database.

3)      Analytics reporting database.

4)      Search Administration database.

First and foremost, is to understand the Crawl components interaction, here I will talk about 3 main components:

a)      Crawl Component

b)      Content Processing component

c)       Crawl database

n  Crawl component: The crawl component is responsible for crawling content sources. It delivers crawled items – both the actual content as well as their associated metadata – to the content processing component.

The crawl component invokes connectors or protocol handlers that interact with content sources to retrieve data. Multiple crawl components can be deployed to crawl simultaneously.

Note: The crawl component uses one or more crawl databases to temporarily store information about crawled items and to track crawl history.

n  Crawl database: The crawl database contains detailed tracking and historical information about crawled items. This database holds information such as the last crawl time, the last crawl ID and the type of update during the last crawl.

n  Content processing component: The content processing component is placed between the crawl component and the index component. It processes crawled items and feeds these items to the index component.

The content processing component transforms crawled items into artifacts that can be included in the search index by carrying out operations such as document parsing and property mapping.

Both the content processing component and the query processing component perform linguistics processing. Examples of linguistics processing during content processing are language detection and entity extraction.· The content processing component writes information about links and URLs to the link database.

 Below shows the dependency data flow between Crawl and Index components:

Crawl Component àStore à Crawl DB àFetch/Process crawled items

à Content processing àcontent processing & extractionà Index Component

Second, once the data has been processed to Index Component, The next step contains other 3 main components are interacting to provide search capability in SharePoint:

a)      Index Component

b)      Index Partition

c)       Query Processing Component

 n  Index component: An index component is the logical representation of an index replica.

In the search architecture, you have to provision one index component for each index replica.

The index component receives processed items from the content processing component and writes those items to an index file.

The index component receives queries from the query processing component and provides results sets in return.

Queries are sent to the index replicas through the query processing component. The system routes and load balances the incoming queries to the index replicas.

n  Index partition: An index partition is a logical portion of the entire search index.

The search index is the aggregation of all index partitions.

n  Query processing component: The query processing component is between the search front-end and the index component.

The query processing component analyzes and processes search queries and results.

Both the query processing component and the content processing component perform linguistics processing. Examples of linguistics processing during query processing are word-breaking and stemming.

When the query processing component receives a query from the search front-end, it analyzes and processes the query to attempt to optimize precision, recall, and relevancy. The processed query is then submitted to the index component.

The index component returns a result set based on the processed query back to the query processing component, which in turn processes that result set before sending it back to the search front-end.
There are my thoughts on the main Search interacting components in SharePoint 2013, Drop me a line if you have any additions or questions.

Thursday, June 13, 2013

How to set parameters' values to SSIS packages


This post to walkthrough step-by-step how to pass parameters to a SSIS package.
Usually you have set of parameters and you want to set the values at run time before the package runs.

The methodology that i will implement in this post, is to export the parameters that i want to set from an XML file by defining the SSIS configuration file using Visual Studio BIDS 2008/2010 or SQL Data tools.

Let's say, you have a SSIS package with a parameter called @myParameter, Here is how to setup the configuration file for the SSIS so we can pass it when the package runs.

1) Open yout SSIS package.
2) Right click after you open the package and select on "Package Configurations".


3) Set the configuration file name and click on Next button.

4) Under Variables node, select the parameter name "@myParameter" you want to export to the configuration file, then click on Next button.
5) Write a name for the configuration and click on next.
6) The VS BIDS will create a XML file with the exported parameter properties.
7) open "TestPackage.dtsConfig" using the notepad and set the value property for the exported parameter.
8) Change the value of the parameter value in the dtsConfig and then save the file.
9) Run your package and the SSIS package will get the updated value from the xml file.

Hope this helps.


Wednesday, June 12, 2013

Windows Phone 8 Get Started and in-depth topics - TechEd NA 2013

Hi everyone,

If you haven't attended TechEd North America 2013 and you are a windows phone developer, This blog post is for you.

I was looking to see all windows phone 8 sessions so i can check them on my own time, I found this blog post that has all videos for windows phone 8 in TechEd 2013.

Take a look and enjoy.


Tuesday, June 04, 2013

SharePoint 2013 REST Service Integration versus Client Object Model

Hi Folks,
Microsoft has released REST service in SharePoint 2013. As a SharePoint solution architect/developer you will be having more than one option to manipulate SharePoint data in your client applications.
With the REST interface in SharePoint 2013, folks are asking about what are the differences? And what are the benefits over each other? I will list all my insights in points in this blog post.
1) One of the main differences between Client Object Model API (COM) than REST API that it requires SharePoint Client assemblies that's only available for .NET technologies. While using the REST API doesn't require any .NET assemblies to be referenced in your client application but use any programming language to execute standard HTTP requests. For example: use the REST API to access SharePoint 2013 from your PHP website or iPhone/Android App.
2) What if I want to consume SharePoint 2013 data with Mobile Applications?
If you are planning to consume SharePoint 2013 data through using iOS or Android devices, then the REST API is the way to go. While if you are developing SharePoint 2013 solution on a windows phone 8 then still you have 2 options either REST or Client Object Model while I recommend to go with the REST for portability and code-reuse purposes specially if you plan to have different version of your application in iOS & Android platforms.
3) Benefits of using REST over COM in Mobile apps? The size of your application will be smaller when you use REST since you don’t need to add any assemblies in your App.
4) What about Windows 8 Apps? If you are developing HTML5 & JavaScript Windows 8 App this requires using the REST API in SharePoint 2013.
5) How do you Access SharePoint 2013 REST API?
The REST API is a WCF services hosted on the SharePoint Application server and has HTTP endpoint; you can access it through the following URL:
If you want to access a site collection:  http://MySite/site/_api/
If you want to access a web site:  http://MySite/site/_api/web/
To do the basic CRUD operations, check out this article:
“Get started with the SharePoint 2013 REST service”
“Using the SharePoint 2013 REST service”
6) When you read the above MSDN articles, the sample requires you to get an access token to access SharePoint 2013 REST API, Is there an alternative?
The answer is yes, you can access the REST API without getting an access token by using the Cross-Domain JavaScript Library. Here is your complete reference to code this.
7) The REST API is not just available for Site Collections and Site scope CRUD operations but it is also available on BCS, Search, and Social Features in SharePoint 2013. Check out these urls:
a) Using BCS REST service:
            b) Using Search REST service:
c) Using Social Features REST service:
As a solution Architect and Web guy, I’m so excited with the SharePoint 2013 REST API. It is a great shift to the web standards and to eliminate the boundaries to integrate SharePoint with Non-Microsoft platforms and tools.
I see this API will be primarily and heavily used by Mobile Developers and System integrators toward SharePoint Hybrid Solutions.
Hope this helps.