Secrets of SharePoint 2010 Exposed at TechEd 2009

In case you aren't aware, the details of SharePoint 2010 are under tight wraps. If you asked any of the presenters at Tech Ed the typical response was "No idea what you're talking about." But that doesn't mean presenters didn't occasionally slip up or say more than they probably should have.

Publically Stated SharePoint 2010 Info

Here’s what has been publically stated about SharePoint 2010 thus far:

• Requires at least Windows Server 2008 64 bit
• Requires SQL Server 64 bit (2005 or 2008)
• Will not support for IE 6
• Public testing to occur first half of next year

This was announced at TechEd by Tom Rizzo, director of Microsoft SharePoint Server, as posted in this InfoWorld article and also on the SharePoint blog.

New SharePoint 2010 Secrets

An exciting slip up that has received no press thus far was from the presentation: Microsoft Visual Studio 2010 Overview for the Business Application Developer. If you check 1:00:33 of the presentation you’ll see the first glimpse of a SharePoint 2010 Site Home Page:

I could tell something wasn’t right when I saw this part of the demo, but fellow Near Infinity employee Joe Ferner put it together first when he saw the big red "Give Feedback" button in the top right. He tweeted about what came next:

"Sharepoint 2010 to have inline list ajaxy editing."

It looks really nice, and is very exciting since SharePoint is currently a little unwieldy. Check out this screenshot from 1:01:08:

So that was the most exciting slip up. Other exciting tidbits were:

SharePoint designer will support saving workflows to re-use for provisioning

I learned this one during the Q&A session of the "Automate Business Processes Using InfoPath Forms with Integrated SharePoint Designer Workflows... All without Coding!" talk. It’s currently a pain point that workflows developed in SharePoint designer can’t be reused in the same way workflows developed in Visual Studio can, so this will be a wonderful new feature.

BDC will probably support updating and inserting data

Apparently MOSS only shipped without this feature because it hadn’t been thoroughly tested. That there has been plenty of time to test this feature by now, so it’s a pretty reasonable assumption that it will be in the next version.

Finally this is pure speculation, but I’m guessing:

SharePoint 2010 to support ASP.Net Data Services API

Microsoft supports two AJAX approaches: client side and server side. The server side uses the UpdatePanel control where you code as usual and everything magically gets AJAXized. The client side version is part of the unreleased ASP.Net 4.0, but ties in tightly with JSON provided by the .Net Data Services API that was released in the .Net Framework 3.5 SP1.

The SharePoint team could have used the server side approach, but why not use the latest technology available, speed up UI significantly, and simultaneously allow web part developers a fantastic level of client side AJAX support? If they made the entire SharePoint API available via the .Net Data Services API it would enable some powerful fast AJAX web parts. It would be awesome.

Finally, the only other tidbit isn’t a secret at all, but what I consider to the most exciting feature of Visual Studio 2010: it will support editing SharePoint solutions without editing XML. I hate XML, so that will be wonderful. So are you ready for SharePoint 2010 today? I sure am. Pity about the wait.

Code Access Security Cheat Sheet

I did short presentation on Code Access Security (CAS) a while ago and put together a cheat sheet to help remember the terms and how they fit together. It includes screenshots of the .NET Framework 2.0 Configuration tool (in Control Panel\Administrative Tools) and uses Entity Relationship Diagram (ERD) Notation. I had to pull it up recently and figured others might also find it helplful.

It describes the following terms:

• Permission
• PermissionSet
• Code Group
• Policy Level
• Assembly Instance
• Evidence; and
• Evidence Type

It should print to 8.5 x 11. Click to enlarge:

Hope you find it useful.

Making SharePoint Title a Calculated Column

All SharePoint lists start with a user editable "Title" column that ECB menu's hang off of and that is the default field to display in associated child tables. This works well most of the time. For example if you have a list like Company, you can change the display name of Title to "Company Name," and then Employee records display that field for their parent relationship.

Where this doesn't work well is lists like Employee, where there is no single column that uniquely identifies a row to an end user. You could create a new "Full Name" calculated column and delete the Title column, but making Title a calculated column just feels like a cleaner solution.

I looked into making Title calculated where it is defined in CAML, but it looked pretty messy, and a true calculated column additionally suffers from the limitation that you can’t pull from other lists. The solution I settled on was to populate it in ItemAdding and ItemUpdating. Of course you tie into those methods with something like:

SPList employeeList = site.Lists["Employee List"];
employeeList.EventReceivers.Add(
    SPEventReceiverType.ItemUpdating,
    "[four part assembly name]",
    "Namespace.EmployeeEventReceiver"
    );
employeeList.EventReceivers.Add(
    SPEventReceiverType.ItemAdding,
    "[four part assembly name]",
    "Namespace.EmployeeEventReceiver"
    );

The only reason this is worth documenting is because setting title is a little messy in the event receivers and there is very little documentation on how to do it.

public class EmployeeEventReceiver : SPItemEventReceiver {

  public override void ItemAdding(SPItemEventProperties properties) {
    string fullName = GetFullName(properties);
    properties.AfterProperties["Title"] = fullName;
    base.ItemAdding(properties);
  }

  private static string GetFullName(SPItemEventProperties properties) {
    string firstName = properties.AfterProperties["First_Name"].ToString();
    string middleName = properties.AfterProperties["Middle_Name"].ToString();
    string lastName = properties.AfterProperties["Last_Name"].ToString();
    return Employee.GetFullName(firstName, middleName, lastName);
  }

  public override void ItemUpdating(SPItemEventProperties properties) {
    string fullName = GetFullName(properties);
    string internalTitleName = properties.ListItem
        .Fields["Title"].InternalName;
    properties.AfterProperties[internalTitleName] = fullName;
    base.ItemUpdating(properties);
  }
}

The tricky part is getting the title field by internal name in ItemUpdating vs. getting it normally in ItemAdding. It’s pretty easy once you work through it, but it took me longer than it should have. So hopefully this helps someone somewhere.

An Exciting Future: C# 4.0, Silverlight 3, MVC Dynamic Data, Live Mesh, VS 2010, ...

Last week I attended the Public Sector Developer Conference in Reston, Virginia. Summary: I can barely contain my excitement for just about everything Microsoft is doing right now for software development. In particular the first talk "What I've learned about Visual Studio 2010, .NET Framework 4.0 (and beyond), Silverlight 3 (and beyond)" was so exciting I almost gave a standing ovation.

What Marc learned about Visual Studio 2010, .NET Framework 4.0, Silverlight 3

So believe it or not this talk by Marc Schweigert (who is a fabulous presenter by the way) was a 90 minute distillation of everything that happened at PDC (which sadly I missed). Consequently it flew like a jet engine, gave barely a minute or two to every topic, and required either massive amounts of caffeine or a good amount of prerequisite reading. Fortunately I had both.

To summarize the presentation I'll just give my favorite upcoming technologies and sort it roughly in decreasing level of my excitement:

Technology: ASP.Net MVC Dynamic Data
Overview: Ruby on Rails for .Net
Impressions: As if ASP.Net MVC wasn't cool enough. This looks awesome! I want to try it on a real project.

Technology: ADO.Net Data Services (Astoria)
Overview: Expose LINQ ORM data (e.g. LINQ to Entities) to web services, now with better BLOB support
Impressions: Powerful, exciting (just a little scary)

Technology: Astoria Offline
Overview: Write disconnected apps that sync to ADO.Net Data Services
Impressions: Frikin' awesome. I can't wait to see/write disconnected apps.

Technology: ASP.Net AJAX - jQuery Support
Overview: ASP.Net will abandon its custom "roll your own" JavaScript framework and adopt a leading open source one .
Impressions: Amazing. I couldn't be happier.

Technology: Live Mesh Web Apps
Overview: Offline capable, in browser or on desktop apps (HTML/AJAX or Silverlight) that run "in the mesh"
Impressions: Wow, this is some really cool stuff, I'll definitely be keeping an eye on this technology

Technology: SharePoint Integration
Overview: Build web parts, wsp files, etc directly in VS without an add-on
Impressions: Fantastic! Awesome! This was a major point.

Technology: Task Parallel Library (TPL)
Overview: Simplify parallelization and avoid programming in threads by using delegates
Impressions: This looks absolutely amazing, I can't wait

Technology: Parallel LINQ
Overview: Parallelize LINQ queries
Impressions: So easy, so powerful, so exiting

Technology: In Process Side By Side Model
Overview: Run both 2.0 and 4.0 CLR code in the same process
Impressions: Wow. Minimize existing app upgrade pain; free developers to use new features in old code bases; free language designers more flexibility in refactoring the C# language.

Technology: C# 4.0
Overview: Named optional parameters and dynamic types
Impressions: Named parameters: cool, dynamic types: controversial, but I suspect a very good thing

Technology: F#
Overview: Functional programming language to ship by default with VS
Impressions: I'm playing with this now, it's very interesting, but I'm not prepared to comment yet

Technology: Managed Extensibility Framework
Overview: Massively simplify creating desktop app add-on frameworks.
Impressions: This has the potential to change desktop apps to very light containers that load functionality as necessary. I am hopeful and excited.

Technology: VS 2010 IDE Improvements
Overview: WPF & XAML Editor better, Multi-Monitor Support, Built on WPF
Impressions: Cool, looking forward to trying. I like the dog-food approach, although as I've pointed out earlier dog-food isn't everything.

Technology: ADO.Net Entity Framework V2
Overview: The DBMS agnostic LINQ based ORM now with N-tier improvement, DDL generation, caching
Impressions: DDL generation was interesting, but apparently won't support deltas, that's for another product, maybe Oslo. Caching support looks fabulous.

Technology: WPF
Overview: WPF to have more controls, better usability in VS, and simplify ribbon programming
Impressions: Cool, but .. am I the only person who hates the ribbon?

Technology: Silverlight 3
Overview: 3D & GPU support; will work as well as WPF in VS; H.264 support
Impressions: 3D = Very cool; VS WPF = Cool, although I may still program in XAML; and H dot two sixty what?

Technology: Web Development in VS 2010
Overview: JavaScript intellisence improvements; view-state improvement, Deployment simplified (e.g. Web.Production.Config & setup packages); new controls (chart)
Impressions: Very nice.

Technology: ASP.Net AJAX – Client Templates
Overview: Simplifying AJAX without an UpdatePanel
Impressions: I'm lazy so I'll probably stick with UpdatePanel & server based approach, but I'm very happy to see the new client based focus as an additional option

Technology: Workflow Foundation (WF)
Overview: More controls, faster, better debugging
Impressions: I haven't really had a need for it WF, but the enhancements sound good.

Technology: Oslo
Overview: Create DSLs that do things like help manage databases
Impressions: I fail to see what problem this solves, but Martin Fowler thinks it's cool, so I'll keep an eye on it.

Technology: SQL Data Services
Overview: Basically Amazon S3 (cloud stuff)
Impressions: In case you didn't know I love referential integrity, so I'm not a big fan. What did impress me is Microsoft will merge the LINQ serialization approach with ADO.Net Data Services. Nice benefit to multiple technologies within one company.

Summary

There's so much going on it's a little overwhelming and I'm sure I missed stuff, but you can see why I'm so excited. I vow to never miss a PDC again.  Anyway I hope this post helped clear up the vast amount of exciting new technologies coming out soon.

Write My Rhino Mocks Expect Statement

Mocking multiple calls to a complicated external dependency (using RhinoMocks for instance) can be challenging if you want to return realistic data. But if you have access to the external dependency and it has decent test data why can’t your mocking tool just call the real dependency and give you the exact Expect() and Return() statements that you need to mock it’s real state?

Well, now it can with WriteMyExpectStatement on CodePlex. How much would you expect to pay for this power? $1,000? $100? How about absolutely free! Alright, guess I've been watching too much late night TV. Let’s examine the problem via an example.

Example: Mocking a Database

Suppose you have a Northwind style database with products and orders and you want to re-order products whose inventory is low:

public class NorthwindService {
  ...
  public void ReorderLowInventoryProducts() {
    using (SqlCeConnection cn = NorthwindDao.GetConnection()) {
      NorthwindDao.Connection = cn; // simple IOC
      IEnumerable<Product> products = NorthwindDao.GetLowInventoryProducts();
      // cache lookup tables in memory
      Dictionary<int, Supplier> suppliers = NorthwindDao.GetAllSuppliers()
        .ToDictionary(k => k.SupplierID); // yayyyy, LINQ

      foreach (Product product in products) {
        Supplier supplier = suppliers[product.SupplierId];
        supplier.OrderMoreProduct(product);
      }
    }
}

It’s a simple example, but already there is a dependency between the data of the two calls that need to be mocked. Specifically GetLowInventoryProducts() specifies a SupplierId whose value must be returned by GetAllSuppliers(). This isn’t complicated enough you couldn’t mock it yourself, but you can imagine a more complicated example with multiple calls and multiple data dependencies.

RhinoMocks.PleaseJustWriteMyExpectStatementforMe = true

In order to mock the above using Rhino Mocks you would first need some code like the following:

private static void ReorderLowInventoryProductsMocked() {
  NorthwindService northwindService = new NorthwindService();

  MockRepository repository = new MockRepository();
  NorthwindDao northwindDaoMock = repository.StrictMock<NorthwindDao>();
  northwindService.NorthwindDao = northwindDaoMock;

  // put .Expect() statements here

  repository.ReplayAll();

  northwindService.ReorderLowInventoryProducts();
}

And assuming NorthwindDao’s methods are virtual you’ll get

ExpectationViolationException NorthwindDao.GetConnection(); Expected #0, Actual #1.

But if you let WriteMyExpectStatement at it using the following:

try {
  northwindService.ReorderLowInventoryProducts();
} catch (ExpectationViolationException ex) {
  NorthwindDao northwindDaoReal = new NorthwindDao();
  using (SqlCeConnection cn = northwindDaoReal.GetConnection()) {
    northwindDaoReal.Connection = cn;
    MyExpectStatement.Write(ex, "northwindDaoMock", northwindDaoReal);
  }
}

You’ll literally get an exception that looks like this:

NorthwindDao.GetConnection(); Expected #0, Actual #1.
WriteMyExpectStatement:
SqlCeConnection sqlCeConnection = new SqlCeConnection {ConnectionString = "Data Source=Northwind.sdf;Persist Security Info=False;", }
Expect.Call(northwindDaoMock.GetConnection()).Return(sqlCeConnection);

Cool huh? Now if you paste that code at “// Put .Expect() Statements Here” and run it again you’ll get:

NorthwindDao.GetLowInventoryProducts(); Expected #0, Actual #1.
WriteMyExpectStatement:
IEnumerable products = new List {new Product {ReorderLevel = 17, UnitsInStock = 25, ProductName = "Chang", Discontinued = false, ProductID = 2, SupplierId = 1, }, … };
Expect.Call(northwindDaoMock.GetLowInventoryProducts()).Return(products);

Just keep copying and pasting until all of your expect statements are there and you’re done.

Limitations

WriteMyExpectStatement knows how to generate the code to do most things and it will recurse into any complex objects you throw at it (e.g. notice how it picked up the fields in the Product class that it had no a priori knowledge of). What it can’t do as well is call methods that take complicated parameters. For instance through reflection it needs to call GetLowInventoryProducts() on northwindDaoReal. If we abandoned InversionOfControl (IOC) and had SqlCeConnection be a parameter to GetLowInventoryProducts(), then WriteMyExpectStatement would have absolutely no idea that it would need to call SqlCeConnection.Open() and so would fail. So as long as you primarily use primitives as parameters to the things you mock you should be able to use WriteMyExpectStatement. And if you have bigger requirements let me know (“codeplex@l” + “eerichardson.c” + “om”), I’d be happy to let you in to the project.

Summary

I realize most situations don’t really need code generation for their expect statements, but if for instance you have existing integration tests that you want to convert to mocks, then something like this is essential. It was for me anyway.

Applications Can't Use SharePoint Master Pages

This is the story of stupid SharePoint problem and an ugly, kludgy, and embarrasing solution. On our project we have the need for an application that looks and feels like a SharePoint subsite. Specifically, we need it to inherit from SharePoint's masterpage. But we also need it to be a separate IIS application.

Problem: SharePoint dislikes Applications

We get the following error when we try to dynamically set the MasterPageFile in code:

System.ArgumentException: The virtual path '/_layouts/application.master' maps to another application, which is not allowed.

Several sites say we just can't do what we're trying to do. We considered a Virtual Path Provider, but decided not to even go down that path because SharePoint's master page undoubtedly has hooks into web.config and httpmodules and such, so we came up with the following hack:

Solution: An Ugly SharePoint Text Manipulation Hack

Deploy a nearly blank page in to /_layouts/ (we do this in our WebAppManifest.xml & WebAppSolution.ddf files). In PageBase (that all pages inherit from) we override Render to retrieve the blank page and do text manipulation to insert our content into it. Here's the code:

protected override void Render(HtmlTextWriter writer) {
  HttpWebRequest request = (HttpWebRequest)WebRequest.Create(GetUrlToBlankPage());
  request.Credentials = CredentialCache.DefaultCredentials;
  using (WebResponse webResponse = request.GetResponse()) {
    StreamReader streamReader = new StreamReader(webResponse.GetResponseStream());
    string pageHostHtml = streamReader.ReadToEnd();

    using (StringWriter stringWriter = new StringWriter())
    using (HtmlTextWriter htmlTextWriter = new HtmlTextWriter(stringWriter)) {
      base.Render(htmlTextWriter);
     
      string renderPageHtml = stringWriter.ToString();

      string title = GetTitleFromHtml(renderPageHtml);
      pageHostHtml.Replace("PAGE TITLE", title);

      string content = GetContentFromHtml(renderPageHtml);
     
      pageHostHtml = RemoveFormData(pageHostHtml);
      pageHostHtml = pageHostHtml.Replace("DO NOT MODIFY THIS PAGE", content);
    }
    writer.Write(pageHostHtml);
  }
}

private static string GetContentFromHtml(string html) {
  Match match = Regex.Match(html, "<body>(.*)</body>",
    RegexOptions.IgnoreCase | RegexOptions.Singleline);
  return match.Success ? match.Groups[1].Captures[0].Value : "";
}

private static string GetTitleFromHtml(string html) {
  Match match = Regex.Match(html, "<title>(.*)</title>", RegexOptions.IgnoreCase);
  return match.Success ? match.Groups[1].Captures[0].Value : "";
}

private static string RemoveFormData(string html) {
  string replaced = Regex.Replace(html, "<form[^>]*>", "",,
    RegexOptions.IgnoreCase | RegexOptions.Singleline);
  replaced = replaced.Replace("</form>", "");
  return replaced;
}

Conclusion

I’m sure there are plenty of optimizations that we can do (e.g. caching), but that’s the basic idea. But this solution makes me feel so dirty. So please, please, dear reader, tell me there is a better way. I want to feel clean again.

Eight Miserable TFS Features

I'd prefer to post a positive, happy, or ideally emotion-agnostic technical post, but today Microsoft Visual Studio Team Foundation Server (TFS)'s source control pissed me off one too many times. I could go on for pages, but this list represents the top eight reasons why you should never pick Team Foundation Server for source control (in decreasing order of annoyance).

1. Can't see changes on get latest

If you like to publically humiliate co-workers when they violate coding standards, or even if you just want to keep an eye on what they're doing, then TFS is absolutely not for you. I'm completely spoiled by Tortoise SVN which allows you to

1. Get latest
2. See which files have changed; and
3. Quickly view the diff

I suppose I should be happy TFS at least allows #1.

2. TFS drops Solution Items

Solution Items are files and folders that exist outside of projects, but within solutions. For instance my current project has an etc directory that contains database scripts and such. First of all it would be nice if Visual Studio automatically recognized new items in the etc directory. Ironically you have to "add existing item" even though it already exists. Fine, whatever. But more importantly it would be nice if Visual Studio didn't periodically delete said files from the solution. As convenient of a feature as that may sound, it has caused numerous problems. A "Blame" has determined that nearly every person on the project has activated this feature leading me to believe it is truly a bug. Brilliant.

3. Checked In Files Are Marked Read-only

Suppose you have a file you want to open outside of Visual Studio (gasp). With TFS if you don't already have Visual Studio open you have to start the IDE in order to check the file out to remove the read only bit. So ten minutes later you're ready to go. At least TFS does allow multiple check-out.

4. TFS server name in project file

TFS embeds the server name in every single one of your project files. The beauty of this approach becomes evident when one team member uses an IP address, while another uses the domain name or computer name. You end up fighting over the project files and automatically checking them out when all you want to do is view them.

5. Can't Access Source Control Outside of Visual Studio

Want to check in or get latest without opening Visual Studio? Yea, you can't. Hope you don't have any java (non-Microsoft) code on your project.

6. Reconciling, Painfully Slow

TFS actually does a decent job of auto merging conflicting files. And when it can't auto-merge conflicting files you can view each file side by side and click which change you want to keep. But the process is painful and slow. For instance if you forget to get latest before check in and there are conflicts you can resolve them, but your check in will always fail. Get latest seems to take much longer when there are conflicts. Auto merge is really slow. If there are un-reconcilable conflicts hitting the "resolve" button takes forever and must be done for each and every conflict. It's like it downloads the entire history of the file. And that wait is mandatory even if you simply want to discard either your or the server's changes.

7. *.vsmdi files

This is just a frustrating known bug where if you use TFS for unit testing (Team Foundation Server Test System) and TFS for source control, Visual Studio will either take this .vsmdi file which is essential file for unit testing and either duplicate it or corrupt it. There's a lot of ink spilled on this vsmdi bug and there are some workarounds, but it's a pain.

8. Moving a folder opens all files in the folder

This was the last straw and the impetus for writing this post. Today I wanted to move a "solution item" into a project. I opened the Source control Explorer (which is impossible to find in the first place) and right clicked, selected move, entered my destination, and TFS in its wisdom opened every file in the folder nearly bringing my machine to its knees. There are hundreds of small annoyances like this every day with TFS.

In Summary

TFS actually has some nice features that I like over SVN. Like the merge tool is pretty good, and being able to see all people who are currently working on a file is nice. But overall the cons heavily outweigh the pros. And if it's any indication how bad TFS is for source control (or how much developers hate it) Codeplex has even implemented a SVN to TFS bridge on their server.

It's ridiculous to me that Microsoft uses this tool internally and sells it and people buy it! I must be missing something. If so please post comments to this post. Because the entire thing boggles my mind.