This installment is perhaps the last on CDM and provides a nice compact document that explains how CDM really works (as of 5.5 – when the underlying architecture was completely revamped). Again, this  is a guide that I created for my own sake. I documented one piece at a time, as I understood its workings and offering this for the first time in public domain for reference. I know there are a lot of big companies still on BMC IdM (Control SA + CDM + …..). So – if any of you oth there run into this blog by Googling CDM – I can assure you that this is the best damn guide on CDM period.

Enjoy

Download (PDF, 1.65MB)

It is a nice workflow product that was acquired from French Company Calendra by BMC when they were into building their IdM arsenal way back in 2004. The product is very good. I’d go so far as to say that it was the best workflow tool in the “Bundled with Identity Management” category.  I used it when architecting solutions for customers using BMC Identity Management and fell in love with it. It had its idiosyncrhasies. It is definitely not the best of breed among workflow tools, in fact it will not even come close to solutions like Lombardi, it is not standards based (no WfMC, no xPDL, no BPMN), but it was tailored for a particular job (Identity related workflows) and did the job well.

It is my habit to use a tool in the RIGHT way and in the best possible manner. And so I did the same with CDM and applied them successfully at countless BMC IdM deployments. Shown below is a document that sumarrizes the best practices with the tool

Download (PDF, 466.84KB)

Combining that UML centric/domain driven idea with existing VIDT and deriving inspiration from Naked Objects (and similar) framework, I came up with a novel concept for next generation VIDT. This approach will take VIDT a level higher and would solve additional problems that VIDT cannot solve as is. I developed the approach on my own time – based on my understanding of Sun IdM, VIDT, Java, JavaEE and of course OO. I proposed it to the Sun management, the idea was accpeted, but nobody had the commitment to take a goose laid golden eggs (meaning billable – that is me) and put on a  project that would take the VIDT tool to next level. The idea was that apart from travelling every week and working insance hours, I could as well contribute to developing the product. That was too much to chew at once.

Anyway, a more detailed explanation of this approach will be provided at a later date. For now, enjoy the presentation below. It is chock full of ideas.

Download (PDF, 676.87KB)

The bootcamp methodology for teaching Sun IdM was too much a like a cookbook. Instead of getting to the fundamentals and concepts and building it from there, the approach was – Here is a button, here are few drop downs, Clikc them and you will get activesync., Oh by the way provide a proxy admin for ActiveSync, but remember not to attach a user form to proxy admin. Oh and here is how you create a deferred task scanner. This approach was totally unappetizing for me. I am a JavaEE developer and architect  by profession and understanding concepts, capturing the domain in UML lay at the center of whatever I did for my customers.  I thought – IdM should be no different. Hence – after a few painful iterations of following the cookbook, I decided to dig under the hood – look at the database created by IdM, analyze the data and come up with how IdM is logically under the hood. This was of course UML (No surprises there – huh ?)

So, while other wannabe “IdM architects” were mugging how to do each task via the cookbook approach, I went on a tangent and approached the domain from common sense approach. As I uncovered each relation between various IdM components – I drew UML on paper. After I had captured enough of those relations, I converted them into electronic format. A cookbook would only address a few pre-defined questions, but my UML was helping me address all sorts of questions that customers posed.  With just a week of bootcamp and a two weeks of self learning using the aforementioned approach, I emerged as a real productive IdM developer in my first project, implemented it very well and was architecting more IdM solutions. I was productive on ground real fast.

Download All UML Diagrams as a ZIP

And here I am, providing a few of those UML class diagrams as samples for you to understand Sun IdM from grounds up using concepts than cookbook. Use these as starting point to build, extend and share your understanding.  As the old adage goes – A picture is worth thousand words. Whoever said it had UML on their mind. So, each of these diagrams captures a wealth of relationships between components and you could look at each of those relationship and conclude what configuration changes are needed for a given task. Observe the relations, navigabilities, multiplcities, dependencies and inheritance depicted to get a deeper level of Sun IdM understanding

IdM Top Level View

Login Module Class Diagram

Relation between IdM Forms, Rules, View Handler and XPRESS

Relations between Waveset User, Account and password policies, Forms, Resource assignments etc.

Relation between a Waveset User, Resource, Reconcilation, ActiveSync, scheduling, admins and forms

Relation between User, Resource, Admin Role, Admin Group

As you can see this material is not meant for glossing over. Rather it is meant for focussed study with deep introspection. Meant for serious readers only

Enjoy and provide feedback. And if you contribute, dont forget to provide me a link to your UMLs

In today’s blog, Iwill  explain the IdM configuration object mayhem in a typical IdM lifecycle and VIDT in greater details. In my next blog, I will layout some uncommon ways of approaching and understanding Sun IdM using UML. (I hope my readers come from a OO background and know Java and UML)

Abstract

Sun Identity Manager deployment experiences have added the maturity to the delivery model. VIDT is the outcome of thus gained maturity. VIDT captures several best practices in IdM implementation and almost productizes them to help jumpstart a IdM project. It is possible that VIDT workings can appear confusing on the surface. This blog probes beneath the surface to reveal why it is, what it is and how it works.

In the process, the blog series will provide a overview of VIDT logical and physical architecture, role of VIDT in a IdM deployment lifecycle, problems solved by VIDT and new challenges introduced in a IdM project using VIDT and how to address them. Future blogs will also cover the patterns and best practices that are implemented by VIDT as time permits.

1. Identity Management Project Lifecycle

This section covers the Sun identity Manager in the context of execution of a Identity Management project. Most importantly, it highlights a few logistical problems faced in Identity Management project with Sun Identity Manager.

1.1 Sun Identity Manager Overview

Sun Identity Manager is the granddaddy of configuration. Sun Identity Manager 7.1 has 50 different types of so-called IdM objects. Each type of object can have 5 subtypes and up to 100 instances. Identity Manager projects tend to be configuration heavy as most of the features of the product are exposed and customized via the IdM-object configuration. Each IdM object is essentially a xml snippet (conforming to waveset dtd) and stored as a single record in IdM repository. Throughout this document any references to xml snippet, IdM object, or object are interchangeable.

Sun IdM itself ships with thousands of objects. Some of them are internal and core to IdM, while others are essentially meant to be customized. A lot of functionality can also be configured via the administrative interface. Even the configurations from an administrative interface translate into xml snippets. Figure 1. shows this.

Any project life cycle involves building a baseline in an environment. Then it has go through the iterative cycles of configuration, development, testing and deployment in multiple environments. test and then deploy to several environments including production. The automation of this process makes it defined and repeatable. IdM projects are no exception. One thing that especially stands out in IdM project is that they involve heavy configuration. Configuration done via the administrative interface involves manual intervention and prone to human errors. The solution for this is to “create the configuration once and deploy everywhere”. Sun IdM provides the hooks to automate the deployment of the se objects. Creation and testing of the objects forms the rest of (also bulk of) Sun IdM implementation.

1.2 Where is my XML?

As noted earlier, Sun IdM implementation is mostly about configuring and deploying the xml snippets into the IdM repository. The xml snippets can be put together by hand or using a logical interface like Business Process Editor (BPE) or the newer NetBeans IdM plugin. In either case, they are imported into IdM repository through a ant build/deploy (The xmls can also be indivdually imported into repository via a special “import exchange file mechanism, but the environment specific parameter replacement will not occur). Many xml objects share a logical relationship because they are part of a single business requirement. However the xmls snippet validation is limited to basic dtd. Complex type based relationships are not completely enforced at the creation time(Netbeans and BPE do some type based validation at creation time). At runtime, the association between the xml snippets is resolved and the IdM tries to get all the defined objects and connect them together to execute the defined business requirement. In doing so, it may fail only at runtime. In addition, the number of xml snippets needed to meet even a modest set of business requirements can be quite large.

The two factors mentioned above overload the IdM implementor with creation of xml snippets (some boilerplate, some slightly customised and some highly customized) during project initiation, which really can be automated. Additionally, considering the fact that IdM is a “sort of” vertical, most of these basic requirements across IdM implementations are more or less the same and need slight adjustment to meet the needs of individual customers. Velocity Identity Deployment Tool (VIDT) was born out of this necessity.

2. What can VIDT do for you?

VIDT is a tool aimed at solving the problem mentioned in the previous section. It is a tool aimed at jump starting the IdM implementation. VIDT lets the business analyst or the implementer to input most of the basic business requirements through a simple point and click interface. A set of xml snippets are created by the tool to meet the business requirements.

In the beginning, the number of xmls generated by VIDT can be a bit overwhelming. However the structure of generated xmls remains same across multiple projects. The generated xml snippets follow several identifiable patterns, best practices and defined process flows. Over time this consistency becomes a familiar territory. Advantages of pre-defined patterns, practices and processes are manifold.

First of all, it cuts down project time by generating the pre-wired xml snippets. It does this by eliminating the need to creating most commonly needed xml snippets from scratch (not to forget the time needed to verify all those are of correct type and correctly linked)

It moves the thought process from xml snippets to actual business use cases. It makes cross-training across projects is easier. Spotting the patterns in a project jumpstarted by VIDT becomes second nature to a VIDT trained implementor. Health checks become easier due to the same reason. Consistent structure can work wonders by making the projects defined and repeatable.

In addition, it also generates valuable documentation automatically for requirements, design, level of effort and Statement of Work. Each of these is a timesaver by itself.

At the beginner level, it is okay to treat VIDT as a blackbox that outputs wonderful xmls based on business requirements. But a deeper knowledge of the tool is necessary to fully exploit its capability and understand when it cannot be used. A deeper knowledge is essential for resolving the logistical and technical challenges that arise with the usage of the tool in a IdM project.

3. VIDT Logical Architecture

Figure 2 shows the high level logical architecture

On the left use cases are entered into VIDT using the UI. Use cases are definitions at pseudo business level – for instance – define a Active Directory resource (easier than IdM console) and a use case might say – I need activesync with Active Directory and voila a bunch of xmls including workflow, forms and Xpress code to support the use case (all based on best practices) get generated. They can be further tweaked then at xml level. Internally the use cases get stored as intermediate xmls using Castor which then generate the real XMLs when requested. Incidentally all this conversions and even the UI is based on Velocity (a open source tool – Find it in Apache) templating. Hence the name VIDT !!

More to come in future blogs

Sun IdM is a very extensible product. Every piece of functionality is loosely coupled.  Everything is configurable in Sun IdM.  Even to the last bit. How is this achieved?

Answer: Configuration Objects

Configuration Objects are pieces of xml to configure a bit of functionality. Large number of such bits come together to define the behavior of the product.  Like communism – the idea is great, but implementation sucks. This is how it causes unthinkable nightmares

1. All configuration objects are xml and stored in database with a id and name.
2. Developer can change any xml object and write back to the database/repository
3. Database will happily accept the xml as long as it is valid and confirms to to the dtd
4. Problem comes during runtime – Another executing object looks for a previously existing xml by name or id. If it does not find it, there is a runtime error. If it finds it, but somebody has changed the behavior of the called object/xml by modifying it and introduced bugs, again it results in  runtime error.
5. IdM consultant’s biggest headache when at a customer site  who has been using IdM for a while is exactly this. One of client’s developer has inadvertantly changed a xml and put it in database. Nobody knows which one. A lot of these xml changes completely skipped the source control and done by just committing the new xml to the production database.
6. Client has changed a xml behavior but retained its original name. During product upgrade all xmls with out-of-the-box names are replaced. Gone are the customizations

With 1000s of xml pieces, it is hard to find which pieces were interacting together before and after and the system is broken. Like I said before, the idea is great, implementation sucks. Tools such as VIDT (Velocity Identity Deployment Tool – Bad name, good implementation) came into being just to address this mayhem a little bit.

In my next blog, I will describe in detail about IdM project life cycle and the importance of VIDT and a logical description of VIDT.

To those whose dont know XPRESS - it is a xml based dynamic scripting/programming language used extensively by Sun IdM

I dont know for sure – but perhaps this language originated as a quick hack language for adminstrators. No hassles of programming, compiling and then redeploying, get new code to run while the Sun IdM is in motion. Sounds like paradise. Indeed .. Until this met real world.

The idea of non compiled xml that does procedural programming is a pain in the neck to get right and get working. Programming by tags means writing code in prefix style. Heck – I’d rather become a compiler myself and write programming in postfix notation

When the rest of the world was moving ahead with Java, .NET and IDE integration, time seemed to be going back for XPRESS. But to their defense, one needs to remember this language was created in 2001 when xml was all the rage from configuration to even cooking Thanksgiving Turkey dinner.

Programming pain aside, the real reason for this bloglet is to show a funny thing about the so called XPRESS. There was joke in private circles that went like this: <b>”The only thing express about the language is its name”</b>. Funny but true.

To prove this, I wrote a simple 5 line program in Java that does a for loop 100 times and printed out the time taken. Then I repeated the process in XPRESS.

Believe it or not – The program written in XPRESS was 1000 times slower (Yep – thats 3 zeroes – Thousand time slower). Even a reflection based Java program is 100 times slower than compiled Java. It looks like the XPRESS code is parsed everytime, converted into some kind of internal syntax (held in Java objects) and then executed by dynamically constructing Java syntax and run as a scriptlet (like Beanshell or Groovy). The whole exercise costs 1000 times more in time. Not to mention the development time for xpress coding.

If one wants to write anything that is non trivial in XPRESS, God save him/her. And to fill this gap some clever guy wrote a tool that converts Java code to XPRESS code. (http://www.xpressutils.com/java2xpress.html) So, one can now write non-trivial code in Java, then run it through this tool, generate the XPRESS code (xml that is), copy the xml into a text file and save it in the database so that the IdM runtime can pick it and execute (Did I mention 1000 times slower) . Goodness gracious!!

One might think, in the world of web services, why doesnt Sun change its best selling IdM by offerring Java APIs and web service to everything from provisioning to workflow. And truth be told, a lot of them exist. But they are not documented correctly or not documented at all. One has to assume the risk and start digging himself/herself and find the right service and Java API. But user is forewarned – Such services are half baked – some features do not exist in the exposed services or Java API. Xpress is the only thing to the rescue in such cases.

On the business/economic front, the XPRESS and arcane workflow interface does not make sense for companies buying Sun IdM either. C’mon start counting – How many XPRESS  ”one trick pony-tailers” are you going to find around you? And contrast that with how many Java developers will you find. It is a irony that Sun – the company that regards itself as a leader in open source has dug themselves into a rut with this product.

And everytime I look at Gartner report I cringe – because they have so many good things to be said about this product – especially how easy it is to build and deploy IdM solution with this product. True, there are a lot of good things abut the product.  Rapid and Easy/clean deployment  with customization is not one of them. Only if they could wrap the strong inner core with a open interface (Java and web services end to end) and support it, while also continuing to support their XPRESS…

That plan never existed at Sun. But alas… Now, Oracle has acquired Sun now and Sun-IdM is planned for sunset. And I can see a few XPRESS cowboys riding into oblivion too unless they retrain themseleves with some mainstream language.

PS: For those who think I came down too harsh on Sun IdM – face it – I have called a spade – a spade. I did a objective review of the interface exposed to the IdM users (users means Sun IdM consultants, IT departments of companies using Sun IdM).

Its not Sun’s product alone that is messed up like this – Have you heard of another IdM product called BMC IdM?  To deal with that monster you will have to know UNIX shell scripting, Windows programming, Top class Java programming, JSF, SDO, Tcl/Tk, JavaEE deployment and debugging knowledge, . The spectrum of language skills needed to do implement a complete BMC IdM shows how many different products were acquired and stitched together. Consequently there is no one consultant that I know of that could do all of these pieces. A few might exist, I dont know what their hourly rate is

And here is a disclaimer – I am ex-Sun, ex-BMC and worked on both Sun IdM and BMC IdM.  I was with BMC 5 years ago and with Sun until 2 years ago. Having seen the best and worst of both products, I am exactly the person that can give the true picture by shredding the glossy picture painted by sales. And I think I just did that.

The benefits of Identity Management (IdM) are well known. Several vendor products rule the roost when it comes to IdM. However in reality, the costs of the vendor products, subsequent enhancements, customization and maintenance does not justify the Return On Investment (ROI) for Small & medium Businesses (SMBs). This blog series takes a contrarian view of the current state of Identity Management especially for SMBs and defines a strategy to leverage your company’s existing IT investment in SOA to achieve Identity Management.

Selling points for traditional Identity Management products

Traditionally Identity Management products IdM products are sold to Information Security division. Security management and security administrators are the primary users of IdM. These primary customers (Information Security division) are not developers by background. Hence the following arguments are used to sell IdM products:

1. Companies that don’t use IdM product use a collection of vast array of scripts for achieving similar results with varying degree of success. A company’s IdM strategy cannot rely on hotch potch collection of scripts because
   1. Scripts are tactical
   2. What if scripts fails,
   3. What if scheduled script doesn’t run – who is notified etc.
   4. Scripts need programming and constant maintenance cost and man power
2. Management needs Reports on user provisioning, password resets etc. and report generation is not trivial
3. IdM vendor products come with
   1. Canned schema (ldap or db)
   2. Canned reports
   3. Canned provisioning workflows
4. Products have great UI capabilities – A lot of provisioning, reconciliation and reporting can be done with the click of a button.

Capitalizing on the audience background, the scripting nightmare stories enhance the FUD factor. The “click of a button” provisioning and reporting demos convince the buyers beyond any doubt. However one needs to look beyond the sales demo to get to the whole truth.

 The whole truth about IdM products

Sure. all those selling points have some truth in them. But can they justify the cost of Identity Management products or the ROI on them – especially for SMBs?

Lets delve into the internals of any Identity Management product in a generic way. I am not stereotyping, however a majority of IdM product architecture goes like this.

1. All IdM products need substantial customization to meet real world needs and this is where the “click of a button” paradigm falls apart. Canned schema, canned provisioning workflows and canned reports can only go so far. Once you reach this threshold, your luck runs out. (which certainly will, during the requirements analysis and design stage itself by the way)
2. Reports, provisioning workflow and other facilities are accomplished as vendor supported extensions on top of basic datastore schema
3. All additional data fields to meet custom business logic have to be crammed into limited extensible space in the tables or by extending the ldap schema. When you extend vendor datastore schema, vendor support is dropped as a general rule.
4. User provisioning, deprovisioning, system access administration each require workflow that integrates with various IT systems in the company to create ids, grant access – logic ranging from simple to complex to sometimes weird, but all have logical explanations in the history of evolution of the company. However each IdM product comes with its own workflow syntax and semantics that is completely different from all other workflow systems in your company and requires integration with all other systems to come up with this logic. Since this kind of integration is expensive for the vendor, they take the route of copying the data from existing systems. This makes the vendor PS implementation easier but in the process, they are setting up operations nightmare down the road for data synchronization and data integrity. This approach is very shortsighted.
5. Add to this the fact that most IdM customizations are written in a propeitory language that is arcane and known only to a few hundred folks (or thousand at best) in the entire world. These folks are expensive, often work for the vendor or their partners (Disclaimer: I am one of them. And yet, truth should be told). You have to go back to the vendor for enhancements. Even your trained folks quit – what do you do? (Note: Even when Java is supported, it is very propreitory API. BTW, this is the best case scenario today.)
6. Needless to say that a lot of these IdM products are built upon their legacy propreitory foundations, acquired products patched up together and not standards compliant.
7. Products cost are justified and sold by their UI capabilities, which is nice facade to the underlying mechanism described above.
8. You now have in your hands, a solution that is isolated from other systems, with only tactical integration at best, that has to be maintained by the security department not to mention the redundant IT infrastructure needed to run the IdM system, including but not limited to clustering, load balancing and troubleshooting!

Now you as a company end up with a solution that

1. Resistant to customizations or collapses under customizations
2. Need tweaks to your working processes to work within the realms of the product,
3. Risk of paying too much to get any useful functionality relevant to your company
4. No support for extensions and your hands tied without any other alternative.

This doesnt mean that Identity management product has no value. It does of course. However keep in mind that:

All you really wanted from the Identity Management product/solution is the core identity management capability for a reasonable price.

Yet the price is inflated to incorporate all the bells and whistles that you really don’t need

Sounds like you got a raw deal! So, is there a better approach? Sure there is. I will discuss this in my future blogs in this series.