Mostafa's Blog

Tuesday, March 08, 2016

Build micro-services in Azure Cloud Services

Hi All,

I have published a new video to cover how to build scalable micro-services in Azure using cloud services. The video covers who to use web and worker roles in cloud services. In addition when to use worker roles vs. web jobs for backend jobs.

Video Details: "Build Intelligent Cloud Services using Azure"

The session covers how to build web, mobile, Api solutions using Microsoft Azure platform. The session covers in detail how to build intelligent solutions using Azure Cloud Services. The session is a demo driven and demonstrate how to design and provision complete end-to-end solutions using Microsoft intelligent cloud.

Session Objectives & Takeaways

Benefits of creating micro services in the cloud using Azure cloud services
Showing End-To-End Use case for building cloud service with web & worker roles with service bus integration

GitHub Repo:

https://github.com/melzoghbi/AzureCloudServices

Enjoy!

Saturday, February 27, 2016

Azure Scheduler fails to trigger scheduled WebJob published with WebApp

Hi All,

I am writing this blog post to show how to fix Azure scheduler fails to kicks off webjobs in Azure webapp while when you try to run webjobs through the portal it works.

I have a web app that has couple of webjobs in my solution. While adding these webjobs to my web app, I configured these two webjobs to run on different schedules. After i deploy the web app to Azure, I found out that the Azure Scheduler fails to run the second web job but not the first webjob!!

After carefully checking all the configs for both webjobs, I found this was the error that is being thrown by the Azure Scheduler for my second webjob:

Unauthorized: Access is denied due to invalid credentials.

Detailed error message:

Http Action - Response from host 'MYWEBSITEwebjobs.scm.azurewebsites.net': 'Unauthorized' Response Headers: Date: Sat, 27 Feb 2016 12:02:01 GMT
Server: Microsoft-IIS/8.0
WWW-Authenticate: Basic realm="site"
Body: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/> <title>401 - Unauthorized: Access is denied due to invalid credentials.</title> <style type="text/css">  </style> </head> <body> <div id="header"><h1>Server Error</h1></div> <div id="content"> <div cla

The solution is the first webjob has the authorization header while the second doesn't. This probably is an issue with VS2015 deployment with Azure SDK.

So you just need to copy these header values from the first webjob to the second one and wait until the scheduler kicks off and things will be working as expected.

Screenshot of the two header values that i have added to my second webjob in Azure Portal scheduler settings:

Enjoy!

Sunday, February 21, 2016

Build Big Data Solutions in Azure using HDInsight

Hi All,

I am posting this post to share my recent presentation on building big data solutions in Azure.

The session covers how to get started to build big data solutions in Azure. Azure provides different Hadoop clusters for Hadoop ecosystem. The session covers the basic understanding of HDInsight clusters including: Apache Hadoop, HBase, Storm and Spark. The session covers how to integrate with HDInsight in .NET using different Hadoop integration frameworks and libraries. The session is a jump start for engineers and DBAs with RDBMS experience who are looking for a jump start working and developing Hadoop solutions. The session is a demo driven and will cover the basics of Hadoop open source products.

Big data solutions in azure from Mostafa Elzoghbi

Hope this helps.

Wednesday, February 17, 2016

Build Intelligent Solutions using Azure

Hi All,

Last night i was honored to present to one of our local user groups in DC about how to build intelligent solutions using Azure.

In this session, I was illustrating different scenarios and practical approaches on building microservices that are elastic and resilient into the cloud. I covered with a practical use case on how to build cloud services that have web and worker roles that integrates with services bus queue as an intercommunication channel between web and worker roles. Also, I covered how to add web jobs into web roles into cloud services as well.

The second part of the presentation was covering Azure App services including: web apps, mobile apps, logic apps and api apps. Then , I covered lots of intelligent features that Azure offers to build intelligent solutions from backup automation, continuous integration, traffic routing, troubleshooting features and deployment slots.

Session Description is below:

The session covers how to build web, mobile, Api solutions using Microsoft Azure platform. The session covers in detail how to build intelligent solutions using Azure App Service. Azure App Service includes web apps, logic apps and mobile apps services as PaaS solutions for engineers. The session is a demo driven and demonstrate how to design and provision complete end-to-end solutions using Microsoft intelligent cloud.

Build intelligent solutions using ms azure from Mostafa Elzoghbi

Case Manager Demo code is on GitHub: https://github.com/melzoghbi/AzureCloudServices

At the end of the event, I had some giveaways that are IoT sensor Kits and "Fundamentals of Azure" books. Most audience asked about how to get the pdf of this book, here is the download link:

Fundamentals of Azure book download link

Hope this helps.

Thursday, February 11, 2016

MySQL Data Source not appearing in Visual Studio 2015 - A complete solution

Hi All,

I was trying to upgrade my solution in VS 2015 Community Edition to use latest MySQL .NET connector with VS integration tools.

After i upgraded my provider to version 6.9.8. I was not able to see MySQL provider as a data source when i run Reverse Engineer Code First tool for EF to refresh my classes with my latest DB changes.

First, If you can't see entity framework when you right click on your project. Make sure that you have installed EF Power tools for VS 2015. Here is the link to download it since Microsoft has not officially released VS 2015 release of this tool.

**Entity Framework Power tools for Visual Studio 2015: download link

Once you install this tool, You should be able to view entity framework option with Reverse Engineer Code first option on your project.

Second thing, you have to see MySql data source so you can connect to MySql db and refresh your classes. below screen shot shows MySql data source.

To be able to to see MySQL data source as shown above, you need two components to be able to connect to MySQL db from VS 2015:

1) MySQL Connector/Net
2) MySQL for Visual Studio

You can get them both by installing MySQL installer that allow you to installer any MySQL components into your machine.

Tip: After installing these two components, you need to close and re-open your VS 2015 to get to see the new loaded MySQL data source.

Enjoy!

Friday, February 05, 2016

Mistakes that kill startups lessons learned

In one of my office hours that i hold every month, I selected this topic to cover and cover the most reasons why startups fail.

In this presentation i covered the following:

1) Startup Structure: How to have the right number of founders, Allocation time and resources, hiring.

2) Idea & Market: How to select a good idea, determine market size and value prop of your product.

3) Raising Capital: How to raise the right amount of money for your startup and how to avoid spending too much.

4) Communication & Execution: How to execute your strategy, how to build healthy relationship with your customers and partners.

My presentation:

Mistakes that kill startups from Mostafa Elzoghbi

Hope this helps.

Friday, January 22, 2016

Application Error net::ERR_CONNECTION_REFUSED in Visual Studio Emulator

Hi All,

While i ran into an issue while running my Cordova App in the Visual Studio Emulator. After logging in to my office 365 tenant, I was getting the following error:

Application Error net::ERR_CONNECTION_REFUSED in Visual Studio Emulator

Given i have all app configuration have been set properly in Azure AD and my settings file in my cordova app (app is o365 connected).

The solution is to use Ripple emulator since it is a browser based emulator that runs under port 4400. So once you switch your emulator to Ripple, you will be able to login to office 365 and redirect back to the redirect url.

Hope this helps.

No authUri provided nor found in O365Auth.authUri

Hi All,

While building an Apache Cordova app to integrate with Office 365 API. I connected my cordova app with office 365 by using Add -> Connected Service option in Visual Studio 2015. When i run my Cordova app try to login to office 365. I am getting the following error:

No authUri provided nor found in O365Auth.authUri

I am using Microsoft.Office365.ClientLib.JS which allow developers to integrate with office 365 services.

The error because the Setting variable is not defined in O365Auth object. So add the follow code in o365auth.js file:

var O365Auth;
(function (O365Auth) {
var Token = (function () ....)();
O365Auth.Token = Token;

// define settings
var Settings = {
clientId: '6de88843-3a72-484c-af8d-3f26d2c3dd52',
authUri: 'https://login.microsoftonline.com/common/',
redirectUri: 'http://localhost:4400/services/office365/redirectTarget.html',
domain: 'MostafaTenant.onmicrosoft.com',
tenantId: '3323232-13e6-232323-a52a-323234423232'
};

O365Auth.Settings = Settings;

Run your project and the rest of your code will pull o365 settings :-)

Hope this helps.

Wednesday, January 13, 2016

Office Add-Ins development best practices

In this video i am covering office add-ins development best practices. The video covers how to provide clear value for Office add-ins, UX principles, market your add-in, publishing strategies, monitor and optimize add-in performance. The video is covering what you need to do from the inception phase of creating an add-in till publishing add-ins to your customers.

If you would like to get to know more about "The basics of office add-ins", i have posted earlier another video for that here:

https://channel9.msdn.com/Blogs/MostafaElzoghbi/Introduction-to-Office-Add-Ins

Enjoy!

Thursday, January 07, 2016

Patterns and Practices in building Office Add-Ins

Hi All,

I'd like to share my presentation on Patterns and Practices in building office add-ins.

I am sharing the best practices to have successful Add-In listing in Office store.
In this presentation i cover the importance of using Fabric UI in developing User Interface elements in Office Add-Ins, Web Extensible Framework (WEF), Napa tools, Development best practices including: Value Prop, Marketing, UX, Optimization technique and security, Identity management in Azure AD, Publishing options, Graph API integration and VS project templates.

PnP in building office add ins - public from Mostafa Elzoghbi

Hope it helps and keep hacking o365!

Tuesday, January 05, 2016

Can't see Office/SharePoint project templates in Visual Studio 2015 Update 1

Hi All,

I am writing this blog post to show how to get Office Developer Tools in Visual Studio 2015 Update 1. I used to have office dev tools and i was able to create Office Add-Ins earlier, but after i installed Update 1 for VS 2015. I am not able to see Office/SharePoint project templates.

When i searched for Office Developer tools, I got this link which is the main link for office dev tools:
https://www.visualstudio.com/en-us/features/office-tools-vs.aspx

After i installed these tools, and then restarted i was not able to see the project templates for office.

I kept searching until i found the solution and the needed components to install on your machine properly.

If you have VS 2015 Update 1, somehow these tools have been removed while installing latest bits of VS 2015 updates.

If you are using Windows 10, From the Cortana search type: Microsoft Web Platform Installer.

Install: Microsoft Office Developer Tools Update 1 for Visual Studio.

Or click on this link: aka.ms/GetLatestOfficeDevTools.

Open VS 2015 and you should be able to view Office/SharePoint project templates.

Enjoy!

Monday, January 04, 2016

Mastering Azure Architecture

Hi All,

I'd like to share a great resource for Architects, Engineers, Business decision managers on how to architect solutions at scale in Microsoft Azure.

The website contains architecture blueprints for architect solutions in 2D and 3D.

The website contains diagrams to build cloud solutions fast and scenarios that have been used to build various solutions in Azure so you can use those in building your solutions.

Website url: https://msdn.microsoft.com/en-us/dn630664.aspx

Enjoy!

Monday, December 21, 2015

Learn R for Data Science Applications and Machine Learning

Hi All,

I am writing this blog post to highlight cool resources to learn programming in R. R language is a widely used programming language for data scientists and engineers who build programmable components in big data solutions.

Here are the resources to get started:

1) Quick-R: the easiest and fastest way to learn R
http://www.statmethods.net/

2) Official CRAN (Comprehensive R Archive Network) resource:
https://cran.r-project.org/manuals.html

3) Introduction to R Programming: Free Course (On-Demand)
https://www.edx.org/course/introduction-r-programming-microsoft-dat204x-0

Hope this helps.

Sunday, December 20, 2015

Apache Spark for developers - part 1

In this blog post i am sharing basic understanding concepts of Apache Spark for developers. My target is to educate developers/engineers with no big data experience on what is Apache Spark and the importance of this platform in working with Hadoop big data solutions.

The target developers should have minimum experience in building business applications or products (desktop, web or mobile) using any OOB language such as: C++. Java or C# developers.

What is Apache Spark?
Apache Spark is a distributed computation framework for big data. It is an open source platform for processing large amount of data in Hadoop ecosystem solutions.

Because it is a distributed platform, there are important concepts to solidify such as:

1) Any spark application contains a driver program which is the main entry point for the application that executes the main function and executes various parallel operations in a cluster.

2) Spark provides a resilient distributed dataset (RDD) which is a collection of data elements that are partitioned across different nodes in a cluster that can be operated on in parallel.

3) We can persist RDD in memory to allow it to be reused efficiently across parallel operations.

4) RDDs can automatically recover from nodes failures.

Tip: To start working with RDDs in Spark, RDD starts with a file in HDFS or any Hadoop supported file system.

5) Spark supports shared variables in parallel operations. There are two types of shared variables in Spark, the first is broadcast variables and the second is accumulators.

You can write Spark applications in Scala, Python and Java programming languages.

To connect with Spark, you need to have a Spark context object which requires a Spark configuration object. The Spark configuration object contains information about your application.

Spark contains Scala and Python shells where you can write and execute your code against Spark cluster nodes.

More to come in the following posts...

- ME

Tuesday, December 15, 2015

Data Science and Machine Learning Training Course for Free

Hi All,

To get started to learn data science and Machine Learning principals i would strongly recommend to try this free course from Microsoft Virtual Academy (MVA) course:

Some key points from the course:

Data science is about using data to make decisions that drive actions.

Data science involves:

Finding data

Acquiring data

Cleaning and transforming data

Understanding relationships in data

Delivering value from data

Predictive analytics is about using past data to predict future values.

Prescriptive analytics is about using those predictions to drive decisions.

Learn more in this course:

https://mva.microsoft.com/en-US/training-courses/data-science-and-machine-learning-essentials-14100?l=UyhoTxWdB_3505050723

Enjoy!

Monday, December 14, 2015

Get Started with Apache Spark Resources in HDInsight

Hi All,

I am writing this blog post to share some important Apache Spark framework for starters topics and foundation understanding points..

Apache Spark is an open-source parallel processing framework that supports in-memory processing to boost the performance of big-data analytic applications.

Spark processing engine is built for speed, ease of use, and sophisticated analytics. Spark's in-memory computation capabilities make it a good choice for iterative algorithms in machine learning and graph computations.

You can write applications in Python, Scala and R in Spark clusters. HDInsight contains out of the box notebooks (tools/Dev IDEs) that allows data scientist to write programs in Spark using:

1) Python language using Jupyter , It also supports R, Julia, and Scala

2) Scala language using Zepplein

Most important libraries for Apache Spark:

1) Spark Sql: A module for structured data manipulation using SQL or DataFrame API.

http://spark.apache.org/sql/

2) Spark Streaming: A module for building stream processing apps the same way your write batch jobs. It supports Java, Scala and Python.

http://spark.apache.org/streaming/

3) MLLib: A library for machine learning

http://spark.apache.org/mllib/

Some useful links and resources for Apache Spark:

1) Apache Spark homepage:

http://spark.apache.org/

2) Learning Python:

https://www.python.org/doc/

3) Learning Scala:

http://www.scala-lang.org/documentation/

HDInsight Apache Spark provides tons of tools out of the box, check out this link to see why would use HDInsight Apache Spark in Azure:

https://azure.microsoft.com/en-us/documentation/articles/hdinsight-apache-spark-overview/

Hope this helps! Enjoy crushing streams of data...

Saturday, December 12, 2015

The best way to test website compatibility issues

Hi All,

I had a chance to attend one of the New York city JavaScript events (JS Open NYC) that was hosted at Microsoft NYC office. I had the opportunity to talk and chat with dozens of open source front end developers about web compatibility and interoperability in modern web design and development.

While i was talking with the event attendees, I introduced new scanning tools that every PM, BA or front end engineer could use to test website compatibility issues.

Microsoft developed tools to scan and test your website for free, below is the website to check all the available tools.

Microsoft Edge Dev Center homepage:
https://dev.windows.com/en-us/microsoft-edge/

In this website, you have four tools to use. I will go through each one of them in this blog post.

1) Quick Scan tool: (my favorite one for technical analysis)
The best tool to do quick scan to your website, it points out all out of date libraries, layouts and things to change in your website to be compatible with most of modern browsers.

Url: https://dev.windows.com/en-us/microsoft-edge/tools/staticscan/

This tool is open source, here is the source code repo in GitHub:
https://github.com/MicrosoftEdge/static-code-scan

2) Browser Screenshots tool: (my favorite one for UX)
This tool shows you how your website looks like in all browser! very handy and useful.

Url: https://dev.windows.com/en-us/microsoft-edge/tools/screenshots/

3) Virtual Machine (VM):
You can download various VMs that have all IE version with different OS options including Linux and Mac. This is a useful tool for internal and intranet sites.

Url: https://dev.windows.com/en-us/microsoft-edge/tools/vms/

4) Remote App:
A way to test your site using Remote App session on any OS you use.

Url: https://dev.windows.com/en-us/microsoft-edge/tools/remote/

I also had a discussion about Chakra (The Core Engine for MS Edge). Since Microsoft announce that the core Edge engine is open source (ChakraCore) and it will be available in GitHub on Jan 2016. Check out the official announcement from Microsoft Edge Dev Team:

https://blogs.windows.com/msedgedev/2015/12/05/open-source-chakra-core/

Hope this helps!

Wednesday, December 09, 2015

Get Started with Apache Storm Resources in HDInsight

Hi All,

I'd like to share some useful resource to get started with Apache Storm in this blog post.
Apache storm is a distributed real-time computational system that allows engineers to process streams of data at scale.

Apache storm is one of the major hadoop ecosystem components, where engineers use it to process the sources of data into hadoop ecosystem.

Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate.

Imagine you need to process endless source of data (such as: Facebook news feed or Twitter feed) and you are going to process this large volume of information and then store in Hadoop. In this case, you want to build a storm application specifying the topology by defining the sources of information (Spouts) and how to process this chunk of data (Bolts).

Every Storm application contains a topology, Set of spouts and bolts in addition to a specification file for the topology.

I compiled some useful resources to get started and work with Apache storm:

1) Apache Storm main website:
http://storm.apache.org/index.html

2) HDInsight Hadoop documentation in Azure:
https://azure.microsoft.com/en-us/documentation/services/hdinsight/

3) SCP.NET, Get started with building .NET apps in C# in Storm:
https://github.com/hdinsight/hdinsight-storm-examples/blob/master/SCPNet-GettingStarted.md

4) Power of Storm with examples:
https://github.com/hdinsight/hdinsight-storm-examples/blob/master/README.md

5) EDX Free online course (Implementing Real-Time Analytics with Hadoop in Azure HDInsight) :
https://www.edx.org/course/implementing-real-time-analytics-hadoop-microsoft-dat202-2x

.
Hope this helps.

Thursday, December 03, 2015

Working with HBase in Azure

I have published a video on how to work with HBase tables in HDInsight HBase cluster. The video is a walk-through on the basics of CRUD operations in HBase.

The video covers the following topics:
1) How to connect to hbase shell tool.
2) How to create tables in HBase.
3) How to select, insert, update, records in HBase.
4) Understanding create, put, delete, deleteall commands in HBase.

The video is giving a basic "Order" table structure as an example and execute all the above operations to it.

Video Link:

https://channel9.msdn.com/Blogs/MostafaElzoghbi/Working-with-HBase-in-Azure

Enjoy!

Tuesday, December 01, 2015

HBase introduction in Azure

Hi,

I have published a new channel9 video about HBase Introduction in Azure.

This video covers an introduction to HBase in Azure. It covers what is HDInsight clusters, What are the available cluster types. What Microsoft Azure offers as Hadoop ecosystem components. The video focuses on HDInsight HBase cluster type and the need for HBase in Hadoop ecosystem to store NoSQL data and the available tools (such as: hbase shell) and commands to use to manipulate data within HBase tables.

The video covers the column families concept for engineers who come from RDBMS background.

This video helps any engineer with no Hadoop experience to understand what is the role of HBase in Hadoop and big data applications.

Channel 9 video url:
https://channel9.msdn.com/Blogs/MostafaElzoghbi/HBase-Introduction-in-Azure

Enjoy!