FirstDoc User Group – Vienna

I’m going to the FirstDoc User Group (FDUG) conference in Vienna, Europe, this year. (For those that are not familiar with FirstDoc, see the links at the bottom of this post).

Every year CSC hold the FirstDoc User Group conference – first in the US, and then in Europe.

I’m looking forward to it. I’ve been ask to present there so it’s time to put the old thinking cap on, and come up with an interesting way of presenting information. (I don’t want to bore people).

The agenda for the Europe conference hasn’t been posted yet, but the one for the US conference has.

The keynote speech will cover CSC’s Long Term Product Strategy. This will be interesting, as the ECM world is very much a lively, ever-changing thing at the moment, as each large EMC vendor morphs, and adapts to meet the ever-changing environment bought about by such things as SharePoint 2010, and cloud computing.

Next on the schedule is a case study – “Global Deployment”.  This will also be interesting as international companies are, and have been for awhile, looking at the challenges of multiple sites, located in disparate locations around the world. The challenges don’t just include the hardware side of distributed systems, but also taxonomies and metadata (ensuring that everyone uses the same vocabulary), etc.

In the afternoon, there will be a panel discussion by representatives of some of the large Pharmaceutical companies on Collaboration, and SPX. SPX is CSC’s technology that allows users to interact with their FirstDoc system from SharePoint. It consists of two parts – SPX web parts, and Wingspan’s DocWay server component that resides on a web service server (see my earlier post for details on this).
I’ve been involved with this technology for the last 4 years, and I am curious what will be covered here.

Later in the day there is also a discussion on FirstDoc Performance metrics.
Now, this is something that I would be very interested in.  How do you actually measure the performance of a system, especially when there are so many parts involved? For example, if a user is in SharePoint, and they use SPX to access documents that reside in a Documentum docbase, there is so much going on. If performance is poor, how do you actually pinpoint where the bottleneck is? I know that there are ways to get information back on the activities that occur, but this involves making some changes in the configuration, and is not really a simple thing to do. If I was there, this is one session where I would be scribbling notes. (I know – in these days, I should actually be typing notes into my iPad2).

At the end of the first day there will be a User Only session. In the first FDUG conference 2007, this session caused a little bit of concern. The idea was that the users would have a chance to talk frankly with the users about FirstDoc (at that stage FirstDoc was the name of the company also – it was bought by CSC in 2008.) However, the fact that there was someone from FirstDoc present in the room did not engender a feeling of openness. At later conferences this was less of a problem.

On the second day, there are more strategy, and users sessions culminating in product demonstrations.

Naturally there is also a social event planned, and this really gives the attendees the chance to mix, and get to know the others that are using the CSC products. There is an opportunity to share, and learn, from others who may be dealing with, or have dealt with, similar challenges.

Related Posts

Next Post: FDUG – Europe – Review of the Agenda

Is Microsoft a Religious Experience?

A Tweet by @pelujan the other day started me thinking. The tweet was:

I responded to his tweet because I do remember “workflo”. It was something that FileNet developed back in 1985. I admit that this was indeed 10 years before I got into IT (having spent those 10 years doing stuff in laboratories), but I was very aware of it as it played a big part in a lot of their technology.

In fact, my first introduction to ECM was PC Docs, and also FileNet’s early Content Management application “Saros Mezzanine”. This was followed by their Image Management Services application running on an AIX system. It stored scanned images on WORM disks in an OSAR unit, and had a robotic arm jukebox. It was a bloody impressive , but also daunting, system (especially when you are new on the job, and you’ve been told to support this system at a very hostile client site).

Over the years I got more an more involved with FileNet and their products, getting to know the idiosyncrasies of each one. I worked as a consultant, and each client had its own unique requirements, environments, and situations.  Very often I would go home  at the end of the day feeling beaten up.

At the end of 2006 I moved into a position working with Documentum, and quickly after, SharePoint. However, this time, I was the client, and so if something didn’t work, someone else was responsible for “fixing it”. This gave me more time to think about the potential of the systems in terms of the industry I was now working in. I actually went home feeling a lot more relaxed.

Now, the one thing that always struck me, when I was working with FileNet, was that, compared to a Microsoft product, there was not a lot of material available. The majority of what you learnt came about through personal experience. You were on the battle field getting the scars. You felt that you had “earned it”.

Of course, there were forums available, and FileNet themselves had a great store of answers to questions, etc. (I used to trawl their partner site just to pick up nuggets of knowledge). Documentum (now EMC) have the same thing which I still use.

At the end of the last century (gawd – that sounds awful) I got my MCSE, and have kept up to speed with Microsoft technology since then. In 2007 I developed a Portal site that hooked into Documentum, and then, having got some scars with that, I got my SharePoint 2007 certification.

Is Microsoft a Religious experience?

Now I am trying to build up my knowledge of SharePoint 2010. This time I’m trying to take a more business application view of the technology. I did AIIM’s SharePoint Master course, which gives a more “real” view of SP2010, especially with regards to Document Management. (See this post, and this one.) However, I realise that it’s still handy to have the MS certification under my belt, so I am working towards Microsoft SP2010 certification also.

I’m don’t want to pay for a course, and so I’m using the over-abundant resources that can be found on the internet (white papers, MS videos, MS learning material, etc). The more material I cover the more I am aware that the same message is being thrown at me – “how great SharePoint 2010 is”. (I’m not going to get into a discussion regarding this, as this has been covered by multitudes of blogs and forums on the internet).

The fact is I find myself slowly, (and blindingly), convinced. I’ve started chanting the mantra, and doing the dance.

Microsoft has produced so much stuff on their latest “shiny object”. It’s amazing. There books, videos, whitepapers, forums, faqs, technet articles, etc, etc, etc. There is also a conference/user group/gathering for the devout, almost every second week. And there are “evangelists” – people who spread the Word.

Got to admit, I am going to one of these conferences in April – the Best Practices Conference, being held in London (#bpcuk). The US one has just finished, and I was following the tweet stream (#bpc11). The funny thing was – I got to the point where I was “religiously” checking on the progress of the conference, and the activities of the participants (albeit the more “tweetal”  – think of the word “vocal” but in terms of tweeting – amongst them). And I found myself just wishing I was there, wishing I was with these people and seeing, and sharing, what they were. (Quick – slap me!)

I never got this “ecstatic feeling” with FileNet. It was all mud and barbed wire. You were earning your stripes “old school”. And even though I have attended the Documentum user group conferences (Momentum) for a few years now (which is one of the high-points of my year – have only missed one over the last 5 years), I’ve never felt the (illogical, zealot-like) fervour that I am starting to experience now.

Related Links

ECM Noir – Killa Hertz & The Case of the Missing Documents – Part 9

…continued from Part 8 —  [All Episodes]

While Killa and Trudy were waiting for extra memory to be put in the Web Services server, Killa showed Trudy how she could split up the crawl of the documents in the Documentum server into smaller jobs. It had been several weeks since he had heard from her, and Killa planned to make contact again to see how things were going.

It had been a month. I hadn’t heard anything from Trudy, so I gave her a call.
“Killa!” was the first thing I heard after I got put through to her. “I’m really glad you called’ she squeaked over the telephone. “And…” I asked, “how did things go?” “It worked!’ she cried. “After the extra memory was put in, I did as you showed me, and the total number of documents being crawled is almost the same as the number of documents in the Documentum repository!” “Excellent, I’m on my way over to check things out.”

I drove over to the office where Trudy was working. She met me at the door, and led me into where she worked. There was still piles of paper everywhere, and that photo of her dog was still there. Pulling up a spare seat, Trudy logged onto the system. “Look,” she said – “there is 2GB of memory on the Web Services server now”. She showed me the screen, which clearly showed the correct amount of memory.

“Ok, let’s see the crawl log.” Trudy switched to another screen where I could see SharePoint’s crawl log. Looking at the bottom of the screen, I could see that the last full crawl had taken place last week. It looked like it had been successful.

Leaning over Trudy’s shoulder, I grabbed a notepad, and one of the pens in the container on Trudy’s desk. I jotted down the numbers of documents that SharePoint had crawled.

Trudy ctrl-tabbed to a spreadsheet she had open. There was listed the numbers of documents that SharePoint had crawled. And she had listed the counts from running a DQL query to determine the number of each content type in the docbase. Each value was in a different color.

I looked through her numbers. “Looks good kid” I said to her with a smile. “Looks like that problem is fixed”. She swayed back and forth with excitement. “And I’ve confirmed with everyone here. They can all find the documents they are looking for.” she squeaked.

“Good – let me just check it one more time.” I went back to her spreadsheet. Fired up DA, and ran a few more queries. The numbers still looked good. “Well Trudy, there’s not much more to do.” “She looked at me coyly “I guess we should celebrate.”  The look on her face was adorable. Sort of between puppy dog, and baby fur seal. “No need to Trudy. Just doing my job.”

I picked up my hat, swung my jacket over my shoulder, and walked out to the car park. Of all the lawyer firms, in all the cities… Well, it was a good job. I’m glad I could help the dame. I headed in the direction of O’Leary’s.

CMIS – what are the adoption plans for 2011?

I’ve been following the CMIS protocol from when it was a “cool idea” till when it became a ratified protocol, and have been seriously wondering what impact it would have on the ECM world (click here for an earlier post).

Recently Generis held a short survey to gauge the feeling of the industry. I’m not sure how scientific the survey was or how many companies were surveyed. And the resulting whitepaper is…I have to say it…ugly. In any case here is a summary of their findings:

  • Most of the companies that responded to the survey use multiple Content Management Systems
  • 26% of the respondents are looking at moving to a common platform, while 52% have no plans, and will keep the systems they have
  • 40% are planning a new system, and 43% are planning system upgrades. Just under 40% are planning content migration. (Note – each respondent may be planning more than one project next year).
  • CMIS doesn’t seem to play a big part in the projects.
  • 87% state that Usability and UI are critical, while 75% consider Richness of Functionality as a High requirement.

[You can read more about the survey here]

What does this all mean? Well, based on Generis’ survey, it seems that CMIS will not be pushed by the users in 2011.

However, other bloggers have also made comments on the future of CMIS in 2011. Laurence Hart predicted at the end of 2009 that 2010 would not be the year of CMIS. In his predictions he quoted Lee Dallas who said “there needs to be application vendor adoption to really create impact.” True! Especially looking at the survey results from Generis.

So will a vendor step up and “force” the others to follow suit? Many vendors have already put into place CMIS functionality (either in the form of a server, or a client). Microsoft introduced in it SharePoint 2007 (administration pack), and it is included as out-of-the-box in SharePoint 2010. Drupal have been busy. As has Alfresco. IBM (FileNet) introduced support for CMIS in V5.0 of the FileNet Content Manager. EMC promises CMIS support in Documentum 6.7.

It seems that each ECM vendor has been quietly toiling away to support CMIS.

But to what end? Is it a case of “Field of Dreams“, where (to paraphrase) “It has to be built before they come”.


Pie’s blog post: Top Predictions For 2010, and Reflecting on Pie’s 2010 Predictions

Craig Rhinehart’s Post: Top 10 ECM Pet Peeve Predictions for 2011

Find out More about those Red Error Messages in Your SPX Web Part

In the Life Sciences industry, many companies use CSC’s FirstDoc to add a regulatory compliance layer to EMC‘s Documentum.

And, if you are using SharePoint as a Portal solution, CSC also offers web parts that offer 99% of the FirstDoc functionality of the thick client. These are known as SPX web parts. (SPX stands for SharePoint eXperience).

The SPX web parts are built on top of the DocWay UI component that Wingspan offers (refer my earlier post for more details).

I’ve been working with SPX pretty much since the first version, and have often seen red error messages appear in the web part. Some of these error messages are self-explanatory. Others are more cryptic.

To get more information about the error message, use your browser’s “View Source” option. In the HTML that is displayed, there is a large section giving more technical information. Using this, the actual problem can be better identified.

The Wingspan Connection – getting SharePoint & Documentum to talk to each other

This is just a short post.  Just want to show an overview of how Wingspan components  allow a user to access their Documentum documents from SharePoint.

wingspan spx sharepoint documentum firstdoc emc

Taken from Wingspan Documentation

Here you can see that there are two main components:

The DocWay UI is a collection of Web Parts installed on a SharePoint Server .

The DocWay Server comprises two components that are always installed together even though they function independently.

  • The DocWay Web Service provides Search, Content Management, and Workflow services.
  • The DocWay Content Transfer Service (DocWay Transfer Service) provides transfer of content between the user’s desktop and individual Documentum Docbases

So, basically, what happens is:

  • A user logs into their SharePoint site that contains Web Parts supplied by the DocWay UI.
  • These Web Parts display meta-data gathered by the DocWay Server about content stored in the Documentum Docbases.
  • Should the user transfer content between their local storage and a Docbase, the transfer is made by the DocWay Transfer Service, bypassing SharePoint entirely.

Included Web Parts for End Users

  • Home Cabinet
  • Subscriptions
  • Checkouts
  • Recently Accessed Files
  • Inbox
  • My Workflows
  • Virtual Folders
  • Repository Browser
  • DQL Query
  • Object View
  • Search

Included Web Parts for Administrators

  • DocWay System Administrator
  • Menu Designer
  • Component Administration
  • Web Part Group Settings

Included Web Parts for Developers

  • DocWay Diagnostics
  • AJAX Call Viewer
  • HTTP Request Inspector
  • System Information

Wingspan produce several other products that allow integration between Documentum and SharePoint. One of these is eResults that I have posted several times about in this blog (see Tag Cloud).

Related Posts

Momentum Lisbon – Opening, and Technical Keynote.

During the opening and technical Keynote presentations at this year’s Momentum, I sat, and diligently made notes of everything that was said.

When I started writing this Post, I realised that I was give a blow-by-blow account of the session, and that this was leading to a rather be a lot large (and more tedious) than I had originally intended. So – instead of that I will try and summarise the key points of the session. (I’ll get the other post finished later).

In the morning, Mark Lewis gave his vision on the “future”. He also explained that he is no longer President of IIG but the “Chief Strategy Officer”. In the afternoon keynote session, Rick Devenuti, the new President of the group introduced himself, and, more-or-less, repeated Mark’s message in his own way. Then Jeetu Patel, the new Chief Technology Officer, expanded on the message giving more depth to it.

The forecast is for cloud.

EMC have recognised that the world is changing.

  • More and more information is being created, and is becoming richer (media files, etc) and more disperse (multiple locations).
  • Regulations are increasing, requiring the keeping of more and more documents. At the same time, being able to easily locate information is becoming extremely important.
  • The User is changing. The way information is consumed is changing, as well as the expectation of the user.

Technology is also evolving. The latest “wave” includes:

  • cloud computing – the increase use of divergent ways of using information regardless of where a person is, is a driving force behind the adoption of the cloud.
  • Mobile Internet – the way users connect to the internet are diverse, and includes such devices as iPads, smartphones, Kindles, etc, etc. The way the internet is being used is changing – there are new social and collaboration norms.

Mark Lewis who gave the above thoughts, also pointed out that businesses need to be able to survive they must have the ability to change quickly. A new “partnership” is required between the business, and IT, rather than IT just supplying solutions/technology that the users must use.

The message that came through from all the three speakers is that most innovation is happening on the consumer side, and that EMC has to be open to this, and willing to accept, respond to external changes. Their new Mission is to “help customers get maximum leverage through:

  1. Help organisations to reduce risk
  2. Increase Agility,
  3. Lower cost”

A new Information Management Technology stack” (according to EMC) was shown that consisted of three layers:

Jeetu expanded on this by adding that the bottom layer also includes “Federation”. EMC recognises that information may reside in the repositories of different systems. They want to still add value.

Momentum Lisbon – A welcome invitation.

I was invited out for dinner after the Welcome Reception by CSC.

CSC offer a product called FirstDoc that adds a compliance layer to Documentum. As well as that, they have products that allows documents in a  Documentum repository to be exposed in SharePoint, while maintaining 96% of the functionality of FirstDoc.

It was a very pleasant evening, and the meal that was served was wonderful.

Thanks Nigel, Paul, Chris & Jim.

Momentum Lisbon – Welcome Reception.

On the first day, at about 5 o’clock, there is always the “Welcome Reception”. This allows people to wander around the booths, and catch up with the exhibitors. At the same time, there is always a certain amount of entertainment going on, often with a typically “local flavor”. On this night, there were artists painting small clay models, dancers, a lady in red, on stilts, The Stig! (yes – you read that correctly), and there was also typical Portuguese food available.

I really like this event because, over the years, at each Momentum I have attended, I am getting to know more of the partners. It’s always great to catch up with these people in a reasonable “non-rushed” moment.

EMC Momentum – Day 1

I arrived yesterday in Lisbon, Portugal for this year’s EMC Momentum, the event focused around Documentum, as well as presenting EMC’s offering to the world of Content Management.

The previous year’s Momentums have been valuable, offering a chance to get to know the vendors in this space, along with their offerings, see how EMC are adapting themselves to meet the continuously changing landscape of Electronic Content Management, and to meet other customer’s, and hear from their experiences (and pain).

Today (Monday) is Day 1. This day is a day for the vendors to set up, the partners to meet to hear EMC’s has planned for them, and to, also, update yourself at one of the several “Fundamentals” sessions that EMC holds. Tomorrow, is when the event starts in earnest, and there are several keynote speeches. This is where we, the loyal customers, get to hear about EMC’s master vision.

Looking at the conference guide, it appears that EMC will be focussing this year on the cloud, and making content available through diverse clients – smartphone’s, web browser, etc, etc. In keeping with this, there are also sessions that update us with what EMC is doing to work with that annoying new kid on the block, SharePoint.

The week promises many opportunities to get to know more about this industry, and to forge new relationships with other customers of EMC, as well as EMC’s partners.

SharePoint 2010 Training/Certification – A comparison of the Microsoft & AIIM offerings

It’s always useful to have certification from a recognized “authority”. It gives you the ability to transfer your skills & experience into something quantifiable. (However, certification on its own, without some real-world scars, should be, in my opinion, considered as just an “intention” to learn more).

I’ve been working in the world of Document/Content Management for quite a few years now  and have worked with many document management, content publishing, search, static content storage, e-mail archiving, etc, applications from different vendors (PCDocs, Tower Software, FileNet, Documentum)

Then a new kid arrived on the block (SharePoint). Fortunately I was given a fantastic opportunity to work with this product in a few large, international, projects. The longest one took over two years, and definitely gave me some scars.

So I did the next logical thing and got my Microsoft certification for SharePoint 2007. Now that SharePoint 2010 is here, I’d like to get some meaningful certification for that.

My first instinct was to see what was required for the Microsoft certificate for SharePoint 2010. But I was also aware that AIIM had their own SharePoint 2010 training course/certification offering so I took a look at that.

It looked good, but I was curious how it stacked up against the Microsoft certification.

I made a list of comparison items and created a mind map (using Mind Manager)

Note – I’ve created an interactive flash file of this mind map. This can be viewed and downloaded from here.


Below, I give an overview of the two offerings:

AIIM SharePoint Certification

AIIM has three different levels of SharePoint certification – SharePoint Practitioner, SharePoint Specialist, and SharePoint Master. Each build on the previous.It was developed based on the requirements of 37 international companies. For a list of the companies refer to the Master Class Info Sheet (see Reference Links at the end of this post)

Each certification is accompanied by a corresponding training course

  • SharePoint Practitioner – covers concepts and technologies for SharePoint.
  • SharePoint Specialist – covers global best practices for implementing SharePoint and
    complementary solutions. Accomplishment of the Practitioner certificate is a prerequisite.
  • SharePoint Master – As well as the previous two levels of certification, the SharePoint Master level requires planning, designing and implementing a SharePoint project.

The AIIM certification is aimed at the Business Managers, IT Managers, Compliance Officers, Risk Managers, Records Management Professionals, as well as for solution integrators and providers, sales consultants, project managers, and technical staff.

Microsoft SharePoint Certification

Microsoft has four levels of certification – MCTS (Microsoft Certified Technology Specialist), Microsoft Professional (MCITP and MCDP), MCM (Microsoft Certified Master), and MCA (Microsoft Certified Architect. Each level builds on the previous one.

  • MCTS (Microsoft Certified Technology Specialist) – highlights area of expertise and helps validate the knowledge and skills working with an enterprise Office SharePoint Server environment
  • Microsoft Certified Professional – helps demonstrate the ability to use Microsoft SharePoint 2010 to excel in a specific, market-relevant job role.
  • Microsoft Certified Master (MCM) – This is more extensive with whiteboard discussions, demonstrations, and goal-based labs.
  • Microsoft Certified Architect – involves an interview with the MCA Review Board

The Microsoft certificates are aimed at people working with SharePoint either as Administrators, or as Developers.

My take on the two Certificates

The Microsoft offerings, are world-recognized (for good or bad). They are quite in-depth, and there is a plethora of training support material, as well as a large community of Microsoft certified people. It is very technical, and gives a person a great opportunity to get down to the “nuts and bolts” level of the technology, as well as the architecture level.

The AIIM training and certification, is not quite so technical, and focuses more on the use of SharePoint in the business, as well as industry best practices. It is also not quite so well-known as the Microsoft Certification. This may be considered a disadvantage, but at the same time, it may also be seen as an advantage. (Thanks to a deluge of paper-only “certified MS” people, the Microsoft certification lost a bit of credibility for awhile there – something that Microsoft have worked on/are working on to rectify.)

So – different courses for different horses. For a side-by-side comparison of the two, have a look at the mind map that I put together (

Reference Material

Microsoft SharePoint Server Certifications

AIIM SharePoint Certification Program

Discussion on the “value” of AIIM certification (from LinkedIn group)

Unique SharePoint Training Program to Focus on Practical Reality of SharePoint






ECM Noir – Killa Hertz & The Case of the Missing Documents – Part 7

…continued from Part 6 —  [Other Episodes]

Killa Hertz’ friend Mike Budrewski had analysed the eResults logs, and had determined that their was a Java memory error. Killa was investigating further.

“Trudy – I need to look at the web server.” Trudy looked up with a puppy dog look in her eyes. She quickly opened up a new remote session, and logged me onto the web server. “OK”, I said to myself, “somehow this thing is throwing a memory error.” I fired up the task manager. The thing was using a little more memory than normal, but it looked OK.

Suddenly my mobile phone rang. Trudy jumped. The girl was skittery. I answered the phone, and heard Mike’s voice. “Killa, I was able to find some documentation about this eResults application. There’s nothing explicit about the error, but it clearly states  that it requires 2Gigabytes RAM. How much is that thing running?” “1 gig” I replied. I was happy. It looked like an open and  closed case. At the same time I was annoyed. Why the hell was a law firm skimping on things like memory?

I looked over at Trudy – she was busy staring at numbers in a spreadsheet. “Trudy – this server doesn’t have enough memory. What you can you do to get another gig installed?”. She looked up. She wrinkled her nose, and trundled her chair next to mine. Her perfume was clearly set on “Kill” this morning..

“Umm, let’s have a look.” The web service server was a virtual one. That meant that, in principle, it should be easy to increase the memory. “Yes” she said, in that excited voice of hers. “However, I’ll have to let the boss know. We’ll probably need to take the server off-line.” I went and grabbed a coffee while she called her boss.

After 5 minutes, she came into the coffee room. “Sure thing Killa, we can do it tonight.” I put down my cup. “Trudy – how long does it take to crawl all the documents in the docbase?” “Well…” she started. “The last time we did it, it took about a week.” Suddenly, I had the urge to be sitting on a stool at O’Learys with a glass of Jack.

“A week is a long time to see if this is going to work.” Even though I was getting paid by the day, there were still limits.

“Let’s see if we can split up the load.” Trudy’s eyes opened wide. She was a good kid. “Look – you’ve got over 800,000 documents in there. We’ll split up the documents into smaller groups. Then we start a crawl on each group of documents. If this memory increase doesn’t help, and a crawl doesn’t work properly, then it doesn’t mean we have to recrawl all the documents.”

Trudy ran over to her desk, and grabbed a pad of paper (it had roses in the corner of each pad) and a pen. “Let me get this down” she said.

“Ok Trudy, let me show you what needs to be done.”

to be continued…

Part 8

The importance of Real testing

testing acceptance development QA

I was involved in a project for a client once that involved running some updates for a SharePoint-Documentum connector. (The connector was essentially a protocol handler that allowed SharePoint to index documents in a Documentum repository, as well as ensuring security was honored.)

The tech guy doing the upgrade had worked many, many hours. The job itself was not complex, but the server on which the application was installed was also used for other things and so, there were starts and stops as others demanded their own slice of time to do maintenance on other apps running on the system unabated.

Finally Techguy had finished the upgrade. He set up a brand new content source that pointed at the repository, and had generated the correct list of crawled properties (these were generated from the Documentum database). Security was also working fine (the connector translated the Documentum security into Active Directory groups/users).

Techguy ran a crawl of the Documentum repository, and checked that there were no errors. Everything looked fine. He performed a search in SharePoint. Yep, the documents were being returned OK. He did further testing, by physically eyeballing a document in the system, checking the content, and then doing searches on various metadata, as well as content searching. Yep, system worked well.

Techguy even performed Regression Testing, following a scripted Regression Test that he had written. It went through the whole shabang of creating a document, sending in for review, getting it approved, changing the status of the document back to draft, making a modification, again for review, again for approval, until the document was effectively a usable document. Everything checked out beautifully. Techguy proudly confirmed that the system was working fine.

The the users were allowed back onto the system. Within hours tickets issues were being lodged. It seemed that the documents that were being returned in a Search, were not opening. The user would click on a search result, and get greeted with a page not found error.

Techguy was called back in. He started looking at the system. Everything looked OK. He was told to look harder. After much scratching of head, and flicking back and forth between various screens Techguy looked up. “I forgot to set up the web service address properly”.

It turns out that SharePoint was able to communicate with the Document repository OK when it came to doing the indexing, but when a request was made to open a document from the search results, a different mechanism was used. If the address of another server is not correctly entered, then a big fat nothing happens when a user clicks on a search result.

Techguy was just that – a tech guy. He was also a user of the system, but when he wrote the regression test document, he was a tech guy. When he did the upgrade he was a tech guy, and when he did the testing he was a tech guy. And as a tech guy he had focused on the technical side. He had made sure that all the main knobs had been turned, he had made sure that the process of indexing was working fine. He had even made sure that the system was returning search results as expected. The one thing that he hadn’t done was to try and open one of the documents that was returned!

Techguy’s oversight raised a very important point. As well as technical guys doing technical testing, end users are also required to do end-user testing. – because the end user does stupid things that the technical people never expect…they actually use the system.

ECM Noir – Killa Hertz & The Case of the Missing Documents – Part 6

…continued from Part 5 —  [All Episodes]

Killa Hertz had worked through the night with help from Trudy. They had gone through the indexing process. It looked like the answer could be in the eResults log. Killa had sent it to his super-geek friend Mike to see if he could make sense of it.

The alarm clock went off at 8am. Swinging my arm I knocked the thing off the bedside table. Being electric, it just keep beeping. I pull the plug out of the wall.

After leaving Trudy’s office last night, I made a phone call. My friend Mike was awake. I expected that. He liked his internet games. I swung past his place with the CD. Trudy had made sure that there was only the eResults log on it.

Mike invited me to stay while he analysed the log.  His flat was small, and messy, and there was no bourbon. I declined. “Mike – call me in the morning when you have an answer.”

So now – it was morning. Still hot, and as sticky as it was last night. After swallowing two cups of coffee, I headed into Trudy’s office. She was there looking at the system. “Hi Killa!” she squeaked far too enthusiastically. I hate morning people. “Have you heard anything?”. I told her that Mike would call me as soon as he had news for me.

“But you know Trudy, it could be that the system is choking while it’s doing the indexing. Let’s have another look at it.”

She logged onto the system for me, and then let me sit in her chair. I had a look at the Crawler Impact Rules in SharePoint. There were none. I poked around and checked out a few other things. There system was 32 bit. Not the best, but didn’t explain why the crawls were suddenly stopping short. There were a few settings in the registry that could be tweaked to increase the amount of memory used. But, again, no point changing those…yet. I made note of them anyway.

Around 9:30, my cell phone rang. It was Mike. He wanted me to come around.

Knocking on his door, I was met by Mike in the same clothes that he had on the last time I saw him. He was talking fast. Clearly a sign of too many caffeine-loaded energy drinks. I didn’t want to be around when those wore off.

Mike pulled a stool over next to his chair. The computer screen was filled with the error logs. “I looked through the logs, Killa. There’s a hellova lot of information in there. I went through each line. This is a smart app.” I could hear that Mike was impressed. “There are a lot of errors, but they are nothing to be worried about. It looks like the system is just reporting that it couldn’t find certain things. These don’t look like they are causing the crawl to fail. I double-checked them anyway. It took me awhile, but about an hour ago I think I finally pinned it down”

I glanced at Mike. He liked his moment of importance. “So what do ya think it is?”, I asked. Mike continued “Memory” he said.”But their SharePoint system is running fine” I said. “No – not the SharePoint server – it’s a Java error.”

“I need coffee” I said. His response was to thrust a can of energy drink in my hand. It was better than nothing.

I thought back over the process. SharePoint indexed the docs. But that didn’t use Java. The documents were transferred in batches from the Documentum docbase to the SharePoint server first. And this was via a web server that did use Java.

“Mike – I’ve gotta go check something. I’ll call you.” Mike handed me a pile of paper. It was a printout of the error log with the Java error highlighted. “As always – Thanks”.

I arrived back at Trudy’s office. “Trudy – give me access to your web server.”

to be continued…

Part 7

  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Global\Gathering Manager: set DedicatedFilterProcessMemoryQuota = 200000000 Decimal
  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Global\Gathering Manager: set FilterProcessMemoryQuota = 200000000 Decimal

ECM noir – Killa Hertz & The Case of the Missing Documents – Part 5

…continued from Part 4 –  [All Episodes]

Killa Hertz had taken control of the computer. Running a few DQL queries he had been able to determine how many, and what sort of, documents there actually were in the docbase. The number that SharePoint was crawling didn’t match…

After taking a swig of cold coffee, I decided to learn more about eResults. This is a protocol handler that allows SharePoint to talk with Documentum. As well as that, it keeps track of the security of  the documents in Documentum. This ensures that security trimming is applied correctly to the documents returned in the search results.

Looking at the SharePoint’s Search Administration screen, and the configuration screen for eResults I deduced the following:

  • A content source has been set up that points to the Documentum docbase.
  • At regular intervals, SharePoint connects to Documentum using the information defined in the content source.
  • Based on a custom filter, SharePoint retrieves a list of the documents that Documentum has 20,000 at a time.
  • Using a web server as intermediary, SharePoint copies the documents from the docbase to a folder on the Index server.
  • The documents are then crawled, and when each document is finished, it is deleted from the folder.
  • At the same time, that the list of documents is retrieved from Documentum, the Documentum security is also translated so that it matches a corresponding group in AD.

Luckily eResults kept detailed logs about its activities. I opened the latest one. There was a lot of information. I started looking through it.

There were several places where the word “error” appeared in the log. They looked like pretty harmless entries, but I wanted to be sure. I’d have to call in a favour.

Mike Budrewski was an old friend of mine. He was born with a copy of “The Geeks Guide to Being a Geek” in his hands. What Mike didn’t know about technology wasn’t worth knowing. Problem with Mike was, if you didn’t have a keyboard, and a monitor, he didn’t really feel comfortable talking to you. Mike wasn’t really a people person.

I looked at Trudy. She was looking tired. “Let’s call it a night” I said. “Copy these log files onto a CD for me. I’ve got a guy who will look over them tonight, and I should have an answer back tomorrow.”

Trudy looked pleased. It was 3 o’clock in the morning, and the caffeine was starting to wear off.

to be continued…

Part 6

ECM noir – The case of the missing documents – Part 4