Yesterday, the third #ECMJam was held. A lot of people were involved and it was a very interesting discussion about
the place of SharePoint in the world of ECM.
Bryant Duhon was the Jam facilitator. Check out his “Introductory” post here (http://www.aiim.org/community/blogs/expert/ECMjam-SharePoint-and-ECM).
There were a number of Questions that formed the basis of the discussion. These were:
Each question raised some interesting responses.
With regards Question 1, there was a feeling that SharePoint was not quite an ECM application:
Others pointed out that the problem isn’t with what the product, itself, can do, but with the “misunderstanding” of what SharePoint actually is.
Question 2 (SharePoint and Governance) was met with a unaimous response – PLANNING & CONTROL
But someone pointed out:
Question 4 discussed what SharePoint did well, and what it did not do well.
While this question didn’t generate the same discussion as others, there were some interesting comments.
The “does well” comments included:
Whereas, the “does not do well” included:
Question 5 asked “Can SharePoint solve collaboration and DM problems for larger companies as well as for smaller?“
Generally it seemed that while SharePoint was useful for a small company, the administration, and maintenance requirements were too high to make it practical.
For a read of the actual tweet stream, click here (http://www.hashtracking.com/fast-report/?hashtag=ecmjam)
Once upon a time, in a far away land, I was present at a demo that a vendor was giving to the end users of a Document Management System. These are users that had worked with the native client (the end-user application) of the DMS for many years. They knew how to make it sing and dance.
The vendor had worked with this customer for many years, and there was a good relationship. The vendor knew how the customer’s business worked. They knew because they were also the vendor of the Document Management System, and had originally worked with the customer to set up the system to match the customer’s requirements.
So, there we were. In a conference room. A representative of the vendor stood up front . As well as that there were 4 other people from the vendor in the audience – a technical gut, a subject matter expert, some from the vendor’s product development, and a client manager.
We waited in anticipation. The vendor was going to show us new technology that would allow the user to access the Document Management System via SharePoint using a web part. Not only could we access the documents, we would be able to interact with the document, and attach it workflows, etc. And all this via SharePoint. This had great potential. It meant that we could create “work areas” customised to the users’ requirements. And the specialised web parts could be configured to returns documents that meet specific criteria.
One thing I need to point out is that the users were not familiar with SharePoint, and certainly not with the concept of web parts. This was new technology for them.
The vendor’s representative coughed. Everyone went quite. Then the representative (who required no introduction as everyone had worked with him at one stage, or another) explained that the technical guy had created a working system that he would use to introduce the new technology. He hit a button on his laptop, and the overhead screen in the room flashed to life.
And what did we see. The vendor had created a SharePoint site, and on it were more than 10 web parts. In two columns. Each showing objects from the Document Management System in various forms (one web part showed an inbox showing workflow tasks, another was a single-box search web part, one had an extended search facility showing, one was for browsing a tree structure of folders, others had specific queries behind them.
The vendor carried on talking about what a web part is, and what each web part did, and, the eyes of the users started glazing over. It was too much for them. This was new technology, and a new way of working. What the vendor showed was too much at the same time. The users were confused. And you could tell by the body language that the users were against what the vendor was telling them.
During the presentation, the vendor would be describing a specific web part and the functionality that it provided.
Several of the more entrenched users (those who had been doing their job since day one, and were damned good at it) would make comments like “This is not how we do it.”, or “We do things differently here.”
I cringed as the presentation died a quick death. The vendor had not planned properly for this audience. Even the managers in the audience were confused by what was being shown. After the everyone had left I approached the vendor, and got into a discussion with him about what had happened. After much analysis, the following was agreed:
- The vendor hadn’t realized that the technology was so confusing. He works with it every day, and, for him, it was second nature. He had not looked at it from the perspective of his audience.
- Too much was presented at the same time. The vendor should have chosen three web parts that provide the base functionality that matched what the users do on a daily basis. Then, once that had been explained, the other web parts could have been introduced.
- There was no “education” done first. The vendor could have started with a explanation of what the new technology was and how SharePoint and web parts worked.
These are all basic things. New ways of doing things, new technologies need to be introduced gently. The users need to be held by the hand as they are shown. And then step by step. The more the users feel comfortable with something he easier it is to take them to the next step, and the more open they are to making suggestions of their own. This allows them to think innovatively.
But what had the vendor done? Strapped everyone in to their stools and bombarded their senses with new, and different concepts. And at all at the same time.
What was disappointing was that the vendor was no stranger to the customer. As I mentioned above the customer company and the vendor company had worked together for years. The vendor knew what the users did.They knew what the users knew.
The vendor left promising to do a better job next time. That they would definitely take the softly, softly approach. And because they did have a relationship with the customer, that was OK. However, know they had the extra burden of having to re-convince an already resistant audience.
In this post I look at some SharePoint-ECM Integration scenarios.
The AIIM SharePoint Master course material that I am studying at the moment presents 4 scenarios about how SharePoint can be used alongside, or integrated with, traditional ECM systems.
1. External Storage Provider
In this scenario, SharePoint is used to manage indexes, metadata, user presentation, etc, and the ECM application manages content storage/retrieval
2. External Repository of Record
In this, all content is managed in SharePoint, until it is declared a record. Then a copy is pushed into the ECM application, where it can only be accessed by Record Managers. SharePoint provides the user interface where documents are created, and edited. The ECM application handles the security, record retention, etc of the document once it has the status of a record. Content only gets into the ECM app via SharePoint.
In a cooperative scenario, all documents are created in SharePoint, where they can be edited, etc. The ECM system is used to manage and control documents that have the status of a Record. Unlike the External Repository of Record scenario, in the Cooperative scenario, content can only exist in one system at a time.
In this scenario SharePoint acts merely as an interface into the ECM app. All documents are created, and managed there.
Andrew offers 8 scenarios. I won’t regurgitate all of what he has written (you can read the posts yourself – see link at the end of this post), but I do want to summarise his 8 scenarios, and discuss where the AIIM scenarios match. (Andrew has got some really cool images on his post that visually represent each of the 8 possibilities beautifully. I’ll use this as well, but remember, they came from his site :-)
Andrew Chapman’s 8 Reference Architectures
1: Keep Systems Separate, Restrict Usage.
Content is moved manually from SharePoint into the ECM application.
2: Loosely Coupled Solution
Content is moved from SharePoint into the ECM application based on some rule, or event.
3: Use SharePoint as a Portal Container
SharePoint uses Web Parts that allow content from the ECM application to be seen, and at the same time, other Web Parts that allow the user to interact with content in SharePoint.
4: Passive Unification in Web Part
SharePoint contains Web Parts that allow a user to see content from both the SharePoint system, and the ECM system. This is from within the same Web Part. The user is unaware that the documents are located in separate systems.
5: Active Unification
Similar to Architecture 4 except that in this Architecture, the user is able to perform more complex operations with the content (managing versions, attaching objects to versions, etc).
6: Passive Back-end Aggregation
An aggregated view of all the content stored across all libraries in created in the ECM. This aggregated view could then be used to make security decisions, perform risk analysis, monitor file usage, etc.
7: Active Back-end Aggregation
All content is aggregated from SharePoint into the ECM system where it is managed, and controlled.
8: Synchronized, Intelligent, 2-way Shortcutting
As with Architecture 7, all content is transparently moved from SharePoint into the ECM system. However in this scenario, users can still act upon the document directly from SharePoint.
As you can see, Andrew Chapman has put a lot of thought into the various possibilities of SharePoint and tradition ECM systems working together.
Looking at what the AIIM SharePoint course material mentions, and comparing it to Andrew’s various architectures, there are close correlations – the AIIM scenarios match the first four of Andrew’s Architectures, with the last four describing variations on the Portal concept.
Andrew Chapman’s post: Eight Reference Architecture Organizer
- SPX Series – A little bit of history (markjowen.wordpress.com)
Day 2 was the second day of the Practitioner’s part of the course. The day was very similar to the previous day – we covered the course material, which Mr English interspersed with real world examples, along with “Bill’s take” on a particular subject. The members of the class also contributed with their own experiences.
To summarise, on Day 1 we covered:
- Core Capabilities
- Components and Parts
- Records Center
- Document Libraries
- Imaging & Capture
- Report Management
- Forms Design
- Workflow & BPM
- Email Management
- SharePoint & MS Office Integration
On Day 2, we covered:
- Design Elements
- Content types
- Site Provisioning
- Admin & Maintenance.
The Practitioner’s course gave a good overview of the capabilities of SharePoint 2010 within the framework of content and records management. The people attending were made up of consultants, Record Managers, Business Managements and similar. When necessary Bill would delve into the technical realm of specific parts of SharePoint 2010, but this was not frequent as the course was not a technical one.
On Day 3 we started the Specialist course. The class was smaller as several people had only been attending the Practitioner’s part.
The material covered for the Specialist course included:
- Information Gathering
- Business Case
- Documenting Requirements
- Records Management
- Change Management
- Test, train, sustain
Initially I felt that a lot of the material covered in the Specialist course could be used in any ECM decision making process.
However, upon re-reading the material I see that it is applicable to SharePoint, in the sense of deciding whether SharePoint is actually the best solution for the business needs, as well as outlining SharePoint strengths and weaknesses. Many useful assessment and decision making strategies tools are described.
Much of what was covered in this course, was of a “dryer” nature (i.e. more conceptual) than in the previous course, and this would result in a slight drop in the attention, and enthusiasm of everyone.
Because many of Microsoft‘s definitions, or descriptions, do not quite match the global “standard” definitions/descriptions found in the Industry (in Records Management for example), often there would be healthy discussions. The specific functionalities of SharePoint were often questioned and the “intended purpose” of such functionality was debated. These times were really valuable, as everyone present had a good understanding of “real” Records Management.
At the end of the course we were presented with an 8 page Case Study. There were three assignments that, because of their group nature, were to be completed during the course, with a third assignment that needed to be done outside of the course, and then presented to AIIM. This, along with passing an online exam are requirements for achieving SharePoint Master Certification.
I felt that two days was not long enough for this course (especially if done in a class). As mentioned in my post on Day 1, the value of doing such a course in the classroom is the ability to ask questions, get feedback on comments, as well as expanding on topics through describing “real-world” situations. This requires extra time, and often we found we were racing through the material, so that we would have enough time for the Case Study assignments.
However, besides that one comment, I really enjoyed this course, and was happy with the material covered. As mentioned, the AIIM SharePoint Master Class is not a technical course, but one designed to describe the concepts and technologies of SharePoint as well as the best practices for implementing SharePoint. I think the course achieved that.
Investigating an interesting, and unexpected, result from indexing a calculated column in SharePoint at the moment.
My research led me to a Microsoft site with examples of formulas used in Calculated columns.
The link to the site is:
- Microsoft SharePoint Server 2010 Business Intelligence Insights Training (obieosobalu.wordpress.com)
- A Tips Spreadsheet for SharePoint Columns (arnoldit.com)
- SharePoint: On Premises and in the Cloud (arnoldit.com)
- AIIM White Paper on SharePoint Deployment (arnoldit.com)
I was involved in a project for a client once that involved running some updates for a SharePoint-Documentum connector. (The connector was essentially a protocol handler that allowed SharePoint to index documents in a Documentum repository, as well as ensuring security was honored.)
The tech guy doing the upgrade had worked many, many hours. The job itself was not complex, but the server on which the application was installed was also used for other things and so, there were starts and stops as others demanded their own slice of time to do maintenance on other apps running on the system unabatted.
Finally Techguy had finished the upgrade. He set up a brand new content source that pointed at the repository, and had generated the correct list of crawled properties (these were generated from the Documentum database). Security was also working fine (the connector translated the Documentum security into Active Directory groups/users).
Techguy ran a crawl of the Documentum repository, and checked that there were no errors. Everything looked fine. He performed a search in SharePoint. Yep, the documents were being returned OK. He did further testing, by physically eyeballing a document in the system, checking the content, and then doing searches on various metadata, as well as content searching. Yep, system worked well.
Techguy even performed Regression Testing, following a scripted Regression Test that he had written. It went through the whole shabang of creating a document, sending in for review, getting it approved, changing the status of the document back to draft, making a modification, again for review, again for approval, until the document was effectively a usable document. Everything checked out beautifully.Techguy proudly confirmed that the system was working fine.
The the users were allowed back onto the system. Within hours tickets issues were being lodged. It seemed that the documents that were being returned in a Search were not opening. The user would click on a search result, and get greeted with a page not found error.
Techguy was called back in. He started looking at the system. Everything looked OK. He was told to look harder. After much scratching of head, and flicking back and forth between various screens Techguy looked up. “I forgot to set up the web service address properly”.
It turns out that SharePoint was able to communicate with the Document repository OK when it came to doing the indexing, but when a request was made to open a document from the search results, a different mechanism was used. If the address of another server is not correctly entered, then a big fat nothing happens when a user clicks on a search result.
Techguy was just that – a tech guy. He was also a user of the system, but when he wrote the regression test document, he was a tech guy. When he did the upgrade he was a tech guy, and when he did the testing he was a tech guy. And as a tech guy he had focused on the technical side. He had made sure that all the main knobs had been turned, he had made sure that the process of indexing was working fine. He had even made sure that the system was returning search results as expected. The one thing that he hadn’t done was to try and open one of the documents that was returned!
Techguy’s oversight raised a very important point. As well as technical guys for technical testing, end users are also required to do end-user testing. Because the end user does stupid things that the technical people never expect…they actually use the system.
Came across a post via via the other day…
@SharePointBuzz retweeted @JoeShepley about a post that was written by Linda Andrews in response to a post Joe had originally written about SharePoint 2010. (You may have to read that again).
Here’s Joe’s original post: http://joeshepley.wordpress.com/2010/06/08/sharepoint-will-own-ecm/.
And here’s Linda’s response: http://www.doculabs.com/?p=1260.
I read Linda’s post first, and started writing a response. Once finished, I thought it might be prudent to read what Joe had originally said, and tweaked my response slightly. Originally I was going to post my response as a comment on the page of Linda’s post. But then thought “Nah – I’ll post this on my blog to give it the glory that it deserves”…
Here is my contribution to the debate:
Interesting article – thanks.
I’ve been working in the ECM for about 16 years now, having cut my teeth on FileNet, and have worked for the last three years with Documentum (and also SharePoint).
When SharePoint 2003 appeared on the scene, it did not even show on my radar. I was aware of the name, but that was it. When SharePoint 2007 took to the stage, I watched the hype and excitement that it bought with it for the first 6 months, but watched that die quickly. While its strengths definitely didn’t lie with ECM, it did offer a lot to collaboration.
SharePoint 2010, on the other hand, I am treating with a modicum of respect, and I have been looking at the “threat” that it is supposedly bringing with it.
I have read Joe Shepleys original post. He makes some very valid points, and while, in principle, you do too, I’d like to share my own thoughts…
For many companies that already have an existing ECM solution in place, the cost, as you pointed out, of swapping to SharePoint is more a reason not to. To uproot a working system, as well as to migrate the documents is not something undertaken lightly.
However, consider a minus of some of the big ECM products. The cost of licences can be quite hefty. This does make SharePoint attractive (even taking into account the points you have made in Reason #3). Any smart company will try and reduce the cost of something that is considered an overhead. As a result, during times of document management system upgrades, it may be that the move to SharePoint could be worthy of consideration.
And, with that in mind, I would like to reiterate Joe’s Shepley’s closing paragraph, by saying that it is not unreasonable to consider that, for the sake of reducing costs, a change in expectations may also be considered. Analyse the actual business process and, if the cost savings are really worth it, adapt it. Maybe a less complex process, that has been built around the “reduced” functionality that SharePoint has, could be put into place.
I’m not going to make any hard predictions, but, maybe SharePoint will actually start owning more and more of the ECM world…
- AIIM White Paper on SharePoint Deployment (arnoldit.com)
- Identity - a way of uniquely identifying people in the system
- Presence - a way of knowing who is online, available or otherwise nearby
- Relationships - a way of describing how two users in the system are related (e.g. in Flickr, people can be contacts, friends of family)
- Conversations - a way of talking to other people through the system
- Groups - a way of forming communities of interest
- Reputation - a way of knowing the status of other people in the system (who’s a good citizen? who can be trusted?)
- Sharing - a way of sharing things that are meaningful to participants (like photos or videos)
Gene pointed out that each social software system had three or more of these elements (but not necessarily all of the elements).
Using Gene’s list I decided to do a case study where I analyse a fictional document management system, and see how it measures up.
The Company: Wet Cleaver Dry Goods
Background: Wet Cleaver Dry Goods designs and manufactures ready-to-wear clothing for farmers. This includes rain wear, winter clothing, informal dress clothing, hats, gloves, etc.
It has factories in three different countries. It uses an Oracle-based Content Management system to store and manage, clothing designs, as well as operating procedures, sales information, customer feedback, and press releases, etc. Designs and patterns are sensitive and need to be tracked. Operating Procedures need to follow a Review process before being available for use. Press Releases need to be routed to the appropriate managers for sign-off before being released, and customer feedback has to be routed to the appropriate department heads. Security is applied to the documents ensuring that they can only be edited by members of each particular department. Each user has an Active Directory account, and a matching account in the CMS. Exchange is used for e-mails.
To provide users with a more “accessible” interface, SharePoint 2007 has been used to create a Portal. Each department has it’s own site which is populated with special web parts that provides access to the documents in the Oracle-based Content Management system, as well as its native functions.
Each web site has been designed by the IT department, based on discussion with the end-users to meet the “requirements” the department the site is intended for. SharePoint groups have been created for each department and populated with the users’ active directory accounts. Each site is secured so that only members of each department can access the related site, and, to ensure that a consistent look-and-feel is maintained, as well as to reduce support issues, the users do not have the right to create new sites themselves, or to customise the sites (“My Sites”). If users from different departments need to work on a document together, a SharePoint site is created along with a SharePoint document library. The required documents are placed in the document library by the CMS administrators, and specific users are granted access to the site. Further to this, a SharePoint Search Center has been created, and with the use of a special protocol handler, is able to index the contents of the oracle-based CMS. Users, however, are only able to find documents that they have rights to.
A separate SharePoint site has been set up to store FAQs, lists of who is in each department, etc.
Analysis: Does this system have three, or more (or any) of the elements that Gene listed? Lets have a look…
- Identity - In this system, each user needs to be logged into the network to access the Portal. Pass-through authentication is used. Thus, each user can be uniquely identified.
- Presence - Although the user can see that they are logged on (their user name is displayed on the screen), there is no way to know who else is logged into the system at the same time.
- Relationships - The Portal has been designed to provide a slightly easier way of performing the tasks that would normally take place in the CMS. That is the processing of documents. As mentioned above, there is a separate site that lists who is in each department.
- Conversations - When users need to communicate with each other they use Exchange. This is, however, separate from the CMS/Portal.
- Groups - The Portal is strictly controlled. IT can create special sites that meet specific requirements, and then users are granted access on a as-needed basis. The CMS administrators export files out of the CMS into the site’s document library where the users can work on them. While this can be considered as a type of community forming, the fact that it is strictly controlled, and not an ad-hoc process negates this.
- Reputation - Apart from the fact that a list is maintained (on a separate site) of who works in each department, and their positions, there is no way to determine the “reputation” of a particular user (e.g. the person who has created the most operating procedures, or has provided the most valuable feedback during a review process).
- Sharing - The only sharing that occurs is the routing of documents. This is not done in an ad hoc fashion, but is defined by business rules, and pre-defined workflows. As such, there is no sharing.
Something else that Gene had done in his post was to create a social software honeycomb. That is, each element was represented by a hexagon. Then each hexagon was shaded depending on whether the particular system supported the social element.
Looking at the Document Management system of Wet Cleaver Dry Goods, the honeycomb would look like this:
Clearly this system does not contain three, or more, of Gene’s social elements.
In a (much) later post I will present a number of ways this system can be made more “sociable”.
- Why isn’t my SharePoint Environment Social??? – SharePoint … (sharepointjoel.com)
Investigating the best way to monitor SharePoint’s Search processes using Windows Perfmon.
- BA Insight Announces Longitude V4 (arnoldit.com)
- SurfRay: Catching the Crest of the SharePoint Wave (arnoldit.com)