Monday, August 17, 2009

SharePoint: The Wild West of Software Development

Some argue that Microsoft developers lack rigor -- that techniques like unit testing and continuous integration are virtually unheard of in this space. That’s rubbish. I would be shocked if the percentage of developers that track code coverage is significantly different for the .Net or Java spaces (not quite mainstream, but respectable). Mind you I’m talking .Net development in general. SharePoint, now that’s another story.

The shadowy world of SharePoint is one in which source control is a distant memory, where no distinction is drawn between development, test and production environments, and where roaming bandits take over small towns with no fear of the law.

Why is this? How has it come to be? The answer is that for the most part SharePoint treats code and data as one in the same. End users can modify data stores: they can add, delete and modify tables (Lists) and columns (Fields) at will. They can open up SharePoint Designer and modify pages, customize workflows, rob banks, and run local Sherriff’s out of town.

And that’s why end users love SharePoint. It removes the slow, burdensome bureaucracy involved in doling out development resources from large centrally planned corporate or government IT departments and places that power directly in the hands of end users, allowing them to get their jobs done faster and to adapt to change quicker. And that is why I maintain that as software developers SharePoint, or a product like it, is in our future -- like it or not.

But the power given to end users makes rigor and good design extremely hard for developers because the production machine you deployed to last month may look completely different now that you’re ready to redeploy. And Microsoft makes good design even harder with an inflexible API. Critical classes like SPSite (think SqlConnection) contain no public constructor, rendering them completely unmockable (unless you’re willing to spend $450 per developer for some TypeMock Isolator magic). And vital classes like SPList (think DataSet) are marked final, crippling your ability to make nice strongly typed entities in your architectures.

So what is a Type-A, quality minded software developer to do?

The thing to keep in mind is that the challenges that SharePoint places on developers and architects simultaneously make good coding practices harder (but not impossible) while making them more important than ever.

Because that column you thoughtfully added as required may get reset to optional by an end user in production, you need to write more good tests that validate once solid assumptions. Because you can’t inherit SPList, you’ll need to encapsulate it, requiring more code per entity. Because you can’t mock essential SharePoint classes you will need to write more integration tests instead of unit tests. Because you’re writing integration tests your tests will be more fragile and you’ll spend more time fixing the test code than the real code.

And because you’re spending more time focusing on quality while your Wild West, short term focused competition in the SharePoint world slings code together, they will appear faster and better able to get applications out the door. It can be frustrating, because over time your well designed and well tested application will be more maintainable and will cost IT departments less. But end users frequently don’t understand this, so as long term minded SharePoint developers we must focus on educating end users.

Combat the short term focused mindset by keeping everything in .wsp files, storing code in source control, avoiding the temptation to apply fixes in production, using continuous integration (yes, for SharePoint!), and most importantly tracking code coverage (yes, code coverage for SharePoint code!). Put quality metrics in status reports for management to see. Track defects and make your stats publicly available (since you’ll have significantly fewer over time than code slingers, and you want end users asking for stats). In short remind the world that quality counts, even in the shadowy realms of SharePoint. If we work together we can bring law and order to the … um, West.


Danny Wahlquist said...

Excellent post, Lee! I believe that you are right that this enabling of users is our future and it is our responsibility to implement so that they get a safety net as much as possible.

Andrew Woodward said...

Brilliant post, and so refreshing to see people talking about developing 'properly' in SharePoint...

I've been talking about this for some time ( and it's great to see other people with the same views.


Lee Richardson said...

Thank you both for the nice words! is exactly the type of blog I've been missing. Does anyone know any other quality focused SharePoint blogs I should be following? Recently I've been feeling like 'proper' development in SharePoint is an oxymoron. For instance dotnetkicks users seems to shun SharePoint articles. But maybe I've been looking in the wrong places.

Keith Dahlby said...

I don't think it's necessarily that the various .NET aggregators "shun" SharePoint, but rather that the SharePoint community just does not to use them as much. In part because so much of knowing SharePoint falls outside of standard .NET dev, and in part because there are just fewer of us (both authors and readers). Personally, I'm much more inclined to ask Google or check @jthake's Diigo links for content that didn't pass through my reader.

As much fun as it is to pontificate on "proper" development techniques, most SharePoint developers are in over their head and just want any solution to the problem at hand. They can clean it up and test it before the next release, right?

Cheers ~

Tom said...

Your third to last paragraph says it all. Taking the time to write good SharePoint code will save time in the long run. Adapting to a complex server solution hosting your app is no more an excuse for writing quality code than learning the Windows API was.

Russell Giddings said...

I think Keith summed it up perfectly when he said "most SharePoint developers are in over their head and just want any solution to the problem at hand".

SharePoint has raised the bar higher for doing development "properly" yet at the same time lowered the ability required to get a solution out of the door.

Its not an easy job to justify to the business the extra cost of doing things properly on a standard project. As SharePoint has widened the gap that extra cost is even higher. So the standard response that you hear is - "In an ideal world yes we'd love to do things perfectly, but given our position in the current economy we just can't afford it".

This all leads to discussions of technical debt and long term cost at which point management eyes seem to glaze over.

And this is the pain that a developer that understands and wants to do things the correct way has to cope with. Unfortunately, it seem like most SharePoint developers out in the field don't seem to fall in to this category.

dotnetchris said...

"So what is a Type-A, quality minded software developer to do?"


I would never want to work at a place that uses Sharepoint for anything other than stupid document storage and excel workbooking.

Prof. E Bud said...
This comment has been removed by a blog administrator.
Andy said...

SharePoint is like a scammer ebay auction, you bid on new plasma and when it arrives its cardboard box with sock puppets inside

Anonymous said...

Pow Pow! (Noise of sharepoint code firing on server!)

What you behemoths fail to realise is the old 80/20 rule...

Namely that most apps don't need to have the rigour you condone designed into them...

Sure some do but the majority do not!

So what you really should be saying is: Which apps should be rigourously designed and why?

In the 3 months it will take you to answer this question the business drivers of most modern businesses will have changed rendering your initial requirements obsolete...I'll have sent the Sherrif packing and will be in the Saloon drinking whiskey and coding on the production server ready to deploy my 4th app of the 3 month period just mentioned...

Ha ha!