LINQ Architecture

LINQ Beyond Queries

• Fluent APIs
• A readable API : Often uses method chaining to code to read like a sentence

• Functional Validation
• Using lambda expressions for a declarative approach
• Keep the code in a data structure for passive evaluation

• Long object construction could be made readable with Fluent API as shown in following two screen shots.
• Building the Fluent API / Internal DSL
• Heavy use of extension methods

• Expression : When C# compiler sees an Expression, it no longer takes converts that coe into IL rather it generates an Abstract Syntax Tree i.e. Datastructure describing that code.

• Static Reflection : Following code shows a method namely PropertySpecifier
• This method can fetch name of the Property passed to it in a type safe manner. 
• i.e. If it is invoked as "PropertySpecifier(m => m.Title)" then it can fetch string name "Title" from it. 
• This is the technique behind Entity Framework's type safe Include method.
• Expression class can yield rich metadata about a piece of code

• Entity Framework : Class which is used as Entity should have properties as virtual. These are used by Entity Framework to inject proxy objects for change tracking mechanism.

• Repository Pattern
• Makes your data appear as if it is living in Memory
• Purpose is to hide all connection and native command complications of remote data source
• Build custom Repositories (like IRepository)
• Use a built in abstraction (EntityFramework : IObjectSet)

• Unit Of Work Pattern
• Tracks changes made to set of Repositories and commits/reconciles these changes with remote data store
• Built in abstractions (NHibernate : ISession, EntityFrmework : ObjectContext / DatabaseContext)

• Build custom abstractions OR borrow it from underlying technology.
• for e.g. custom IUnitOfWork vs NHibernate : ISession or Entity Framework's ObjectContext
• ObjectContext is harder to test as there is no built in interface.
• If built in abstraction is used then above layers get access to all the features provided by built in abstraction. For e.g. Attach,Detach, Include operations in Entity Framework

• DALayer supporting Composable vs Non-Composable queries
• Composable queries are achieved by returning IQuerable
•  Composable Pros
• Allows Linq Provider to take holistic view of a query and perform bulk of the query in remote data provider
• Generated queries are as efficient as possible
• Works well with small Applications and small databases
• Composable Cons
• Easier to miss an Index, could be very costly in large databases
• Predictable queries. Thus helps database optimization

• Stopping Deferred Execution (aka Lazyness)
• Use one of the greedy operators
• ToList,ToArray, ToDictionary
• Produce a concrete type
• Sum, First, Single, Count

• Deferred Execution 
• Pros
• One may not need result. (Less common scenario)
• Cons
• It doesn't fail fast
• It might execute more than once

• Include method in Entity Framework is used for
• Loads related entities
• Solve (N + 1) Select Problem

• Testing : Mocks or Fakes
• Test doubles for IEnumberable and IQuerable are easy
• For fakes use AsQueryable to convert IEnumberable to IQueryable

• Moq : Open source Mocking Library

Transactions: .Net and WCF

Transaction Architecture

• Resource Manager (RM)
• RM is system,product or any component which manages data participating in transactions
• RM either commits or Rolls back changes
• for e.g. RDBMS, MSMQ or SQL 
• Two types of RMs
• Durable: Resilient to system failures. RMs memorize state of transaction. If system is shut down in between then upon restart Transaction can proceed from its previous state.
• Volatile: Non-resistant to system failures. e.g. This transactional implementation of some core .Net classes.

• Transaction Types
• Long Running
• Long time to complete
• Waiting time for actions and/or messages. i.e. generally waiting for human actions
• Explicit commit & rollbacks (i.e. Compensation)
• No locks are used as it would have severe performance implications
• Atomic
• Take little time to complete
• Implicit commit and rollback
• Locks are acquired to ensure Isolation
• Transaction Protocols
• Decide scope and type of communication between participating applications
• Types of protocols
• Lightweight Protocol
• OleTx Protocol
• WS-Atomic Protocol (WSAT)
• Lightweight Protocol 
• Transaction spans across only one AppDomain
• Only one durable RM
• Multiple volatile RMs
• No cross AppDomain calls (no client-server calls)
• Transactions are NOT allowed to propagate to another application or appDomain
• OleTx Protocol
• Allows transaction to cross AppDomain
• Allows transaction to cross machine boundary calls
• Multiple durable RMs
• Windows Remote Procedure Calls (RPC) only
• No cross platform communcation
• Usually not allowed through firewall
• Typically used in Windows intranet scenario
• Allows transaction propagation to other application or appDomain as long as they all run Windows
• WS-Atomic (WSAT)
• One of the WS-* standards
• Same as OleTx plus interoperable communication
• Transaction can propagate to different (non-Windows) platforms
• Transaction Manager
• Transaction Protocol defines transaction boundaries and communication rules
• Transaction Managers manage transactions practically
• Three TMs
• Lightweight Transaction Manager (LTM)
• Uses Lightweight protocol
• Kernel Resource Manager (KRM)
• Uses Lightweight protocol
• Ability to call on distributed File system (TXF) and distributed registry (TXR)
• Distributed Transaction Manager (DTC)
• Uses either the OleTx or WSAT protocol
• Manages transactions across machine and process boundaries

• Promotion
• Transaction Manager selection happens automatically. i.e. System.Transactions has promotions capability. 
• Image : "1. Transaction - Promotion"
• Not all RMs support promotion
• RMs must implement IPromotableSinglePhaseNotification interface
• MSMQ and SQL Server 2000 doesn't support promotion

What is 2-Phase Commit Protocol

• Ensures Transaction atomicity across multiple RMs
• Each RM running can fail or commit indecently
• 2-Phase Commit Protocol is used to manage distributed transactions
• Image : "2 Transaction - 2 Phase commit.JPG"
• Image : "3 Transaction - DTC with OleTx Protocol"

WCF Transactions

• Transaction Propagation: Allows clients and services to participate in the same atomic transaction
• Not all WCF bindings support transaction propagation 
• When transaction propagates from client to service, DistributedIdentifier for both transaction will be same
• When service operation creates a new transaction (i.e. transaction didn't propagate from client), transaction will first have only LocalIdentifier and DistributedIdentifier will be empty
• Transaction Flow and Transaction Protocols

Image : "4 Transaction - Propagation and Protocol.png"

• Enabling transaction flow 
• Step 1: By default bindings disable transaction flow. To enable transaction propagation WCF client should set TransactionFlow attribute of the binding to true.
• Step 2: Configure transaction flow for Service Operation with attribute TransactionFlow. TransacttionFlow has following options.
• NotAllowed: default. Propagation is not allowed. If client tries to propagate transaction it is ignored, no exception is thrown.
• Allowed: If client want to propagate transaction it is allowed, but transaction propagation is not mandatory. If binding doesn't support propagation and client is trying to propagate then an exception would be thrown.
• Mandatory: Propagation is mandatory.Both client and service must use binding which supports propagation and TransactionFlow attibute must be set. Any violation leads to exception.
• OperationBehavior Attribute
• OperationBehavior attribute could be applied on service operation
• It has two parameters
• TransactionScopeRequired: Wraps service operation in TransactionScope
• TransactionAutoComplete: Invokes transactionScope.Complete() method at the end of service operation
• OperationBehavior is oblivious about source of transaction
• It can be propagated from client
• It can be instantiated from server
• Image : "5 Transaction - Transaction Propagation Configuration.JPG"

Bootstrap

- Introduction to Bootstrap

- Bootstrap is popular open source toolkit for building front end applications.

- Bootstrap helps you build web pages with the help of Html,CSS and JavaScript.

- With Bootstrap one can get good looking web site up and running quickly.

- Bootstrap has 

• Two CSS files. bootstrap.css and bootstrap-responsive.css
• Few png image Sprites. Each image sprite containing multiple glyph images.
• One JavaScript file Bootstrap.js. This file is required only if one is using Bootstrap widgets. This file has dependency on JQuery. So if main page requires Bootstrap then JQuery.js will also have to be included.

- When a Html page has links to multiple CSS files, order matters. If there is a class defined in two or more CSS files then bottom one would be used. i.e. Bottom file has more precedence than upper file.

- Overriding Bootstrap class definitions: Copy definition of the class which is to be overridden. Paste it into another CSS file. Modify the pasted class definition. Make sure that Html page links Bootstrap.css file before and modified CSS file after.

Pluralsight: Web Farms for Developers

- What is a Web Farm

- Web Farm is a Two or More web servers providing same service. It consists of a Load Balancer(Software or Hardware) and two or more Web Servers. A load balancer routes traffic to web servers.

- Benefits of Web Farm: Reliability and Scalability.

- Types of Load Balancers

        * Hardware Load balancer

        * Application Request Routing (ARR): Extension of IIS

        * Windows Network Load Balancing Service (NLB): Comes with Windows Server

- Types of Load Balancers

- NLB: Network Load Balance Manager could be used on Windows Server to setup and configure Web Farms.

- NLB Disadvantage: NLB doesn't operate at HTTP level, it operates at IP level. Thus, HTTP based Server health checks are not possible with this.

- ARR
        * Health Test: On certain schedule go and check that specified web page returns expected result.
        * Client Affinity: Enables Sticky Sessions.

- ARR Drawback: can be used only for HTTP traffic. Can't be used for Mail or other Non-HTTP traffic.

- Some web farms support Sticky sessions using Cookies.

- Web Farm Management

- Every single web server has to be identical.Virtualization is easiest way of ensuring this.

- Staging Server is necessary to deploy code on Web Farm. This is where code and configuration are verified.

- MSDeploy\WebDeploy is utility from Microsoft to perform automated deployments.

Debugging the Web with FireBug, WebDeveloper, and Fiddler

- Debugging Web

- WebKit: Open Source project which powers Chrome,Safari and Firefox
- Firefox has 3D view, which is helpful in debugging layout problems.

- Firefox addon's for developers: Firebug and Web Developer

- Debugging JavaScript (With Firebug)

- Scripts tab doesn't show JavaScript files which have errors in them. Compilation errors are shown in console window. 

- Conditional Breakpoints could be set.

- 'Stop On All Errors'. This option needs to be set, so that Firebug stops on an JavaScript error and shows it in Console.

- Firebug has built in Performance Profiler.

- Debugging CSS (With Firebug)

- Images and color codes could be previewed by moving cursor on it.

- Live Edit: Enables to edit CSS and see its impact. 

- Style panel in HTML Tab. Style Panel has drop down which has a value of ":hover". If this is selected then, Styles panel shows styles applied when mouse hovers over selected element.

- Debugging Network (With Firebug)

- Firebug Net tab shows (At the bottom)
            * Number of HTTP requests sent by browser to load current Page.
            * Size (in KB or MB) of data received. ( Also amount of data fetched from Cache).
            * Time taken to Load entire Page (Also time taken to fire Load event).

- Firebug Net tab shows all the resources downloaded. If an resource is image then hovering over corresponding record will show snapshot of image.

- Firebug Net tab enables user to Disable browser cache.

- Firebug XHR Tab is useful for Testing API calls made by JavaScript. Clicking on a record, shows value returned by that call. One useful feature here is that user can resend the request.

- Firebug Net Tab has Timeline column. The blue vertical line here shows when DOMContentLoaded event (DOM of the Page is loaded) was fired and Red line shows when Load event (Page is loaded. Browser will attempt to show page) was fired.

- console.timestamp(): This will add a new line to Time stamp column.

- What is Fiddler? Advanced HTTP Debugger.
            * Serves as Network Proxy to watch network traffic.
            * Shows detailed network request information.
            * Allows copying and construction of HTTP requests.

- Fiddler allows to create a HTTP request (Which is not possible to do from Browsers). Thus a API call could be tested, even before actual client for the API is written.

HTTP Fundamentals

Chapter 1 : HTTP Resources

- http://food.com/recipes/sandwich

• URL scheme: (for e.g. "http://" in above address). Part before ://, it Defines how to access a particular resource. Other schemes are https,ftp and mailto(mailto://abhijeet.nagre@gmail.com). Everythinng after :// is specific to URL scheme.
• Host: (for e.g. "host.com" in above address). DNS maps the host name to IP Address.
• URL Path:(for e.g. "recipes/sandwich" in above address)

- http://food.com:80/recipes?type=breakfast

• Port: (for e.g. "80" in above address). 80 is default for HTTP and 443 is default for HTTPS. This is the port at which Server is listening.
• Query String: Preceded by "?" (for e.g. "type=breakfast" in above address). It is up to the host application to interpret this.  Mostly used to pass multiple name value pairs separated by &.
• Fragment: Preceded by #. Not processed by server. Handled by client only. Identifies particular element in Html, which client should focus on.

- Having keywords in URL is a good Search engine optimization.

- URL Safe characters: URL can contain only safe characters.
Following are safe characters which cane appear is an URL.
            * Upper case and Lower case letters (a-z AND A-Z)
            * Numbers (0-9)
            * $_-.+*'(),

- URL Encoding:If URL contains unsafe character, it could be percentage (%) encoded (also know as URL Encoding). I.e. replace the character with "%{numeric code of the character in ASCII}". So ! could be sent as "%21" as 21 is it's ASCII code.

- Content Type of Resource: Server mentions type of the content to client. Thus client knows if the resource is image or video or text or something else. Content Types are specified as per MIME standard. 

- Content Negotiation: A resource represented by Single URL can have multiple representations. for e.g for multiple languages (for e.g. same recipe in French, English etc.) or in different formats like Html, MS Word or PDF etc. When client makes a request it can specify media types it can accept. Piece of code written in JavaScript can ask for JSON representation of a resource, whereas a piece of code written in C# can ask for representation in XML for the same resource with same URL.

Chapter 2 : HTTP Messages

- HTTP messages come in pairs (Request message and Response message). Information in the message is all in readable text.

- There are tools which give view of HTTP request coming and going from your computer. Fiddler is one such tool. Most browsers also provide such view.

- First line of the HTTP message (both Request and Response) is always explicit about its intent.

- HTTP Methods

Method	Description
GET	Retrieve a Resource
POST	Update Resource
PUT	Store a Resource
DELETE	Delete a Resource
HEAD	Retrieve the header for a Resource

- Even tough HTTP specification mentions above methods. GET and POST are mostly used methods others are rarely used.

- If you are writing a HTTP web service, you might want to use HTTP Put and Delete methods. Be careful, as there are few server side technologies and pieces of hardware who do not support these messages.

- POST method is used when browser needs to send some information to Server.

- There is part of HTTP specification which describes Safe and Unsafe methods.
- Safe methods let you read a resource from server. This method doesn't modify resources in Server. GET and HEAD fall in these category. Get operation should never have a side effect on Web Server.
- Unsafe methods are the ones which let you change resources on web server.
- If a Html page has Form and contents of the form are already Posted; then Browser will give warning if user Refreshes that page. PUT,POST and DELETE fall in this category.

- Post-Redirect-Get Pattern (aka PGT): When a Post request is processed by Server, the request is redirected to another Page (with HTTP Get command).

- FORM GET vs FORM POST
A Form is generally used with POST method. But it can be used with GET method as well. For e.g.
     
<form action="results.cshtml">
    <input name="q" placeholder="search" type="search" />
    <input type="submit" />
</form>

When above form is submitted a GET message is sent to Server at URL path results.cshtml. The value of input search will be appended to URL. URL will have query string with name q (input box's name). Whereas when contents of Form are sent to Server using POST message, values entered in the input boxes are sent as message and not as Query String.

- HTTP Status Code Categories

Method	Description
100-199	Informational
200-299	Successful
300-399	Redirection
400-499	Client Error
500-599	Server Error

- Fiddler (fiddler2.com)

Chapter 3 : HTTP Connections

- Browser implement HTTP protocol. i.e. Browsers act as HTTP initiating agent and sends HTTP messages using (mostly) TCP.

- TCP: Does Flow control i.e. ensures that sender doesn't send messages too fast for the receiver to process them.

- WireShark : Could be used to do deeper analysis than what Fiddler does. Fiddler shows HTTP messages exchanged between browser and Web Server. WireShark even shows TCP handshakes,shows messages transferred between all the subsequent layers involved in the HTTP message transfer i.e. TCP, IP and Data Link layer.

- If a Web server doesn't allow persistent connections then it must include a header "Connection-Close" in Response. Shared Hosts would generally do this.

Chapter 4 : HTTP Architecture

- URL doesn't mention which HTTP method (GET,POST etc..) is to be used.

- All the information required to complete HTTP transaction is contained in HTTP messages.

- HTTP Proxy: 
Proxy server can
            * Act as Access Control device. e.g. Filter all traffic going to Facebook.com.
            * Strip out confidential data out of HTTP messages.
            * Create Audit Trail on Traffic          

- Forward Proxy: is closer to client than to Web Server.
            * Forward Proxy requires some configuration in client software or Browser.
            * Forward Proxy provides service to some limited set of users. e.g employees of company or users of ISP.

- Reverse Proxy: is closer to Server than to client
            * Completely transparent to client.
            * All the requests coming to the Web Server are coming though Reverse Proxy.
            * Proxy server can reduce load on the Web Server by providing services like Compression, HTTP message logging etc.

- Services provided by Web Proxy Server
            * Load balancing. Some Proxy Servers can look at how much CPU and Memory a server is using and distribute load based on that.
            * SSL acceleration: Encrypt and de-crypt HTTP messages
            * Security: Filter out dangerous HTTP Messages.
            * Caching Proxies: Cache HTTP Response Messages

- Fiddler works by installing itself as Proxy on the machine. Thus it can intercept all HTTP traffic.


- HTTP Headers for Caching 
            * Cache-Control:
            * Expires: Deprecated in HTTP 1.1 but used for backward compatibility
            * Pragma: Deprecated in HTTP 1.1 but used for backward compatibility

- Values for HTTP Cache-Control Header
            * Public: A response for everyone
            * Private: A response for specific user.
            * no-cache: Don't cache
            * no-store: You never saw this response (i.e. Delete message immediately)

- Caching
            * Public Cache: Shared among multiple users. Generally 
                  resides on Proxy Server
            * Private Cache: Web browsers cache HTTP messages marked
                  as Private Cache on users disk. 
                  Internet Explorer stores caches at "Windows\Temporary Internet Files" 
                  location.
                  Chrome's cache files could be found at "chrome://cache/"
            * HTTP GET message is safe message so could be considered for Caching. 
                  PUT,POST and DELETE are unsafe messages so are not considered for Caching.

- ASP.Net Cache Control Headers
            * Response.Cache.SetCacheability(..)
            * Response.Cache.SetExpires(..)

- Client sets "Last-Modified" header in HTTP messages to let Server know if the resource has changed since that. If resource has not changed, server sends HTTP response message with Status code 304. Which means client could use cached copy.

Chapter 5 : HTTP Security

- Some Load balancer's support Sticky Sessions. i.e. HTTP Requests belonging to a session are sent to same server.

- Stateless HTTP enables State management by using Cookies.

- Cookies: Server sends state information to browser using Set-Cookie header. Subsequent requests made by the browser contains this Cookie. 

- Session Cookie vs Persistent Cookie: Session cookie is discarded when Browser is closed whereas Persistent cookies are not discarded when Browser is closed. Persistent Cookie needs to have an Expires value.

- HTTP follows a Challenge Response format for authentication. When client asks for a secure resource, server returns a 401-Unauthorized response, Response also mentions which authentication protocol is used for Authentication. Client then asks credentials(username and password) to user and sends another request to server with credentials (Credentials are sent using WWW-Authentication header. All subsequent requests have WWW-Authentication header, which contains credentials.

- HTTP doesn't dictate how credentials are validated by Server.

- HTTP specification mentions two authentication protocols i.e. Basic and Digest.

- Basic Authentication: Sends username and password to server as Base 64 encoded string (via Authorization header). Thus this is very unsafe and rarely used.

- Digest Authentication: This is similar to Basic Authentication except client doesn't send plain text username and passwords to server. Client applies MD5 hash on username and password and sends result to server. Thus it is not possible for a sniffer to know username and password.

- Forms Authentication: Application has complete control over how authentication is managed.When client requests for a secure resource, Server redirects it to login page using HTTP 302 temporary Redirect. Login page lets user enter credentials, which are POSTed to server.  The response will also set a cookie indicating user is authenticated.

- In Forms Authentication Credentials are sent in plain text, So it is necessary to use HTTPS or Secure HTTP.

- Secure HTTP and HTTPS are same protocols also know as SSL or TLS(Transport Layer Security). Encrypts HTTP messages before they are sent. Uses https scheme in URL instead of regular http scheme. Default port for HTTP scheme is 80 and default port for HTTPS is 443.

- HTTPS adds a layer inbetween Application Layer and Transport Layer. HTTPS requires server to have a cryptographic certificate.This certificate is sent to client during setup of HTTPS connection. Certificate includes Server's host name. Certificates are provided by providers like Verisign. Certificates use public private keys. Administrators have to purchase certificates from certificate providers and install them on server.

- HTTPS encrypts HTTP messages except host name. Everything else (URL path,Cookies,headers, Body) is encrypted. Avoids Session hijacking as no eavesdropper can hijack Session cookie. Client can use certificate to validate(authenticate) host. HTTPS does not authenticate client.So some Authentication mechanism is required to authenticate client. HTTPS makes Client Authentication protocol secure as it encrypts username,password and authentication cookies in the Http messages. Clients cans authenticate by using Client side certificate, but this is rarely used.

- HTTPS Downsides: 
           * Performance: Large sites use specialized hardware called SSL accelerators.
           * Performance Connection setup is longer as additional hand shakes 
                  are required.
           * Can not be used as Public Cache however clients can cache them as 
                 private cache.

- OpenID: is a standard for decentralized authentication. Users do not have to create multiple passwords for various web sites. Also every web site doesn't have to manage authentication, it can delegate authentication management to identity provider. Identity provider stores and validates Identity.



Follow up 
- Architectural Styles and the Design of Network-based Software Architectures
- Sticky Sessions

Play by Play: Scott Allen and Dan Wahlin

- Web Essentials is good plugin available for Visual Studio.
- Boot Strap components: http://getbootstrap.com/components
- Batarang: AngularJS plugin for Chrome