Big Migration equals big headaches

So lets say you have a single site with 50 pages and you are looking to move from Content Management System X to DNN/Evoq.  Easy enough, bring up one site on one monitor and the DNN/Evoq site on another and spend a couple days copy/pasting.  Oh if life were only that simple!

We're looking at moving sites with hundreds to thousands of pages so a manual copy/paste of content just isn't going to do it.  So now I'm writing, from scratch, a migration tool to try and move content from SharePoint 2013 to DNN/Evoq.  I knew going into it that this wouldn't be easy, business make their fortunes writing these types of things, but I have pulled almost every rabbit out of the hat in making this go and I'm coming to one conclusion.

Content Management Systems don't want you to leave so they don't make it easy for you to go.  Now granted in this case getting information out of SharePoint isn't proving nearly as hard as putting information into DNN/EVOQ.  You see the web-based CMS's are designed for you to do everything through the browser and I'm trying to script and code.  Just not what they were made for.

So here is where I have been and where I might go.

1. Export from SharePoint and "call" the "create page" from the DNN/EVOQ API to create the pages...  this failed because it's very difficult because it's not just one API call to create a page and it's not documented AT ALL.
   
2. Export from SharePoint and "insert" into the database directly for DNN/EVOQ.  I'm a database guy so this should have been easy.  Problem is the database seems to be uber-normalized and a single change can sometimes need multiple rounds trips to the same table. So doing it reliably without wrecking the whole system is very tricky
   
3. Export from SharePoint and "create" a "Export" package that DNN/EVOQ can process using their own "Import" wizard to bring in just pages and content.  so far this looks like the most bulletproof method and I'm making baby steps toward it.  I've got the actual file that holds all the "page" info and am just running into trouble with DNN/EVOQ parsing it during the "import" pre-check.
   
4. This may be where I end up, I just found a project that utilizes the new DNNPrompt API.  This API actually has a command called "New Page" and can be called from powershell.  I know that the #3 solution will probably lead to further enhancement and give me more options, but this powershell route MAY be a stopgap to just get going.  If I can't get the parsing of my package to work then this may be where I end up

So the journey continues and should I figure out a path I will be posting my solution so hopefully it will help others regardless of if you're moving toward DNN or Evoq we all need a way to bring in our data and not all of us are blessed with small sites that manual effort is enough.

Let me know your thoughts and if you've come up against this before. Looking forward to hearing your horror and success stories with migrations large and small.

Google translate the monkey executives can't get off their back

Often times we are asked about "translation" or someone just says outright, "put google translate on my site".  They'll even go so far as to reference a law that says they need it in order to comply.  The truth is "translation" isn't about a technology or piece of software, but rather similar to accessibility it's about a set of best-practices and recommended features for your site.  

Certainly as a developer it's much easier to put a few lines of JavaScript into a theme file or include it in the header and just check off the "translation" checkbox, but are we really meeting the goal?

I think it's too much time under the business analyst hat that has me always looking deeper into not only the headline of requirements but why they exist at all.  So when someone tells me they "have" to have their site translated I push back and ask "why". Then when they reference a law or a statute or a policy, i read it and actually ask them if translating the website alone will mean compliance.

Take for example this sentence from a document a user recently brought to my attention...

From: Code of Federal Regulations (Title 29 Part 38)

Excerpt: A recipient should ensure that every program delivery avenue (e.g. electronic, in person, telephonic) conveys in the appropriate languages how an individual may effectively learn about, participate in, and/or access and aid, benefit, service, or training that the recipient provides.

Now if you read that sentence you'll see that translating the website is only a very small portion, in fact it's only a small portion of "electronic". You see because, and I'll use the google widget as an example, these "silver bullet" widgets only translate the visible HTML it means they miss any text "burned in" on images. It also won't cover any PDFs or office documents someone may download.  Let alone emails you may try to exchange with a potential "aid recipient".

Now take into account the fact that your call center and walk-up services also either must have access to or be trained in interpreting and you've now got a picture of what following the true spirit of the law means.  Think of it this way, how well will your baseball be in the field if you only man first base and the pitchers mound and leave all the other positions empty?

Now for those who need further proof to demonstrate that there is no "magic pill" that solves the issue and in fact they can make things worse by trying to have an automated system translate "meaning" and "intent" here are a couple links from federal sites which cover the topic very well and should hold weight with management, at least more so than the blog ramblings of a developer.

 

Lost in Translation - A quick piece about how automated translation is seen as a "silver bullet" but really the problem is much larger.

https://digital.gov/2012/10/01/automated-translation-good-solution-or-not/

 

Dept. of Labor (Machine Translation) slide deck: a comprehensive look at the pitfalls of automated translation and a set of best practices and suggested approved alternatives and additives to automated translations.

https://www.dol.gov/sites/dolgov/files/OASAM/legacy/files/062414Machine_TranslationWebinar.pdf

Of course, I'm always interested in your views on it and your experiences in handling this request.  Please comment and let me know your story and how it went.

Think bigger when thinking about Content Types

This post will be less from a technical perspective and more from a architectural one. I'm talking about a lesson learned and it remains one I'm still struggling with today.  With defined content types it's easy to think of the visualizer first then create a content type to support it and then for rinse and repeat.  Let me give you an example...

1st need (codename: current focus): I need to be able to put 3-4 high priority items on the front page, preferably I'd like to bring attention to them with a graphical header and a bit of text.  Then when clicked on take them to a page with more information. Ideally the client would like to see something like the bootstrap "card" presentation style.

2nd need (codename: article): I need to be able to host a blog/news section on the site. It should include the ability to have a listing index in a multitude of presentation styles and have the details of the "article" be either a pdf/word document or a link on another website or possibly even no further information at all.

So naturally when I started working on these I saw it as two different things, it was easy since it our current solution each was handled through different mechanisms the thought was to continue that thinking by keeping them separate.  Upon further reflection though I realize that they are in fact the same content item and the only difference is how they are exposed to the user.  This is exactly what EVOQ/DNN is headed with liquid content and it's the mind-set you have to wrap your head around.

I can use a single Content type and then just have multiple visualizers, one to show the items as "cards" and the other as a "listing". The trick is going to come down to filtering and tagging when it comes to what is shown where, but the data itself lines up the same on both.

Thus my plan is to build a "Article" Content Type which holds these fields:

• Title [Textbox, also becomes Content Item Name]
• Description [Multi-line plain Textbox]
• Article Date/Time [Date/Time] (this way people can post/pre populate articles)
• More Information Type (this determines where the title links to in the "list" visualizer)
  • Event Details [HTML]
  • Webpage [URL]
  • Document [Asset]
  • None
• Rollup Image [Asset]
• Optional Elements
  • Article Image [Asset]
  • Article Image Location (Top, Bottom, Left, Right)
  • Expiration Date [Date/Time] (this way you can prepopulate when the article should be removed from view]
  • Featured [checkbox]

Then I'll create visualizers that expose the "rollup" versions using various looks like..

• 3 wide cards (roll-up image on top)
• 4 wide cards (roll-up image on top)
• Cards (masonry/pintrest board layout)
• Article Listing (media markup, roll-up image on left)
• Article Listing (media markup, alternate sides roll-up image)
• Article Detail

Meeting Visualizer Troubles (can you hear me NOW)

Background: Government is filled with meetings, we have meetings to talk about upcoming meetings and meetings we just had sometimes we even have meetings to discuss how to avoid other meetings. So I knew that one of my content types was going to have to be "Meeting" or in my case I was going to call it "Event" so it sounded more generic.

In my world a "Event" has certain "must-have" elements...

• Title [Textbox, also becomes Content Item Name]
• Description [Multi-line plain Textbox]
• Begin Date/Time [Date/Time]
• Is it all day? [Radio button yes/no]

Then it has additional optional information...

• End Date/Time [Date/Time]
• Physical Locations (zero or more reference objects of type PhysicalLocation described below)
• Required elements
• Address Title [Textbox, also becomes Content Item Name]
• Street Address1 [Single Line TextBox]
• City [Single Line TextBox]
• State [Single Line TextBox]
• Zip Code [Single Line TextBox]
• Optional Elements
• Street Address2 [Single Line TextBox]
• Rollup Image [Asset]
• Virtual Location Information [Multi-line plain textbox]
• More Information Type (this determines where the title links to in the "list" visualizer)
• Event Details [HTML]
• Webpage [URL]
• Document [Asset]
• None
• Agenda [Asset]
• Minutes [Asset]
• Event Documents [Asset(s)]

This seems like a good starting place and I'll be adding onto it as I go. Now onto my issue.

Issue: Often times there are multiple committees and/or subcommittees as well as we need to display these in a way that my users are accustomed to I need to be able to seperate the "Upcoming" meetings from the "Previous" meetings.

Solution(s):

For the "committee" and "subcommittee" portion I'm going to leverage the "tag" portion on the content item.  Users should be able to start entering the committee and subcommittee name into the tag box and after the first entry of a given committee it will auto-prompt them.  Now this is a little prone to user error, but it gives great flexibility for the names to be added to and deleted over time. The I can just throw visualizers on the page and use the default "filter" to limit which committee is show on a given page.  This should accomplish the need to separate out committee schedules in a way that's fairly easy for the users to manage and me to control.

That was the easy part, the other part is much more complex since the visualizer ONLY allows you to filter by "TAG". Even the advanced URL filtering has trouble doing logical assessments and is more adapt at doing things like "beginswith" or "endswith" type evaluation.  Here I need a "time" evaluation to limit what is showed.

Having nothing at hand I had to get creative and the answer was in creating TWO visualizers, one for "future" events and one for "previous" events. This wasn't ideal but it would work if i could get the visualizer itself to determine which results to show vs not show.  The problem was NOTHING in the documentation for date filters showed a way to compare a datefield to "NOW" or "TODAY". It existed in the Shopify version of liquid but not in the EVOQ version. So after MUCH searching and trial and error I found a couple crumbs that when combined showed me an UNDOCUMENTED ENUM.

By using the code below I'm able to get the current date/time in a formatted manner and assign it to a variable that I CAN then use in an IF statement.

{% assign xNow = 'now' | date: "MM/dd/yyyy" %}

You can add hh:mm:ss to formatting filter to get the time as well, but in my situation I only needed the date.

So now my if statement looks like this to get me my "Upcoming" events.

{% assign xNow = 'now' | date: "MM/dd/yyyy" %}
{% assign xbeginDate = beginDateTime | date: "MM/dd/yyyy" %}
{% if xbeginDate >= xNow %}
...
...
..
{% endif %}

And I'll be darn it works!  Hopefully this saves you some time looking through tones of other sites.

Saving my users through Structured Content

When I first started with EVOQ for my main workplace I decided that I would utalize the Structured Content engine heavily.  Now anyone familiar with DotNetNuke knows that there are a ton of options available for "liquid content" type template development to make life easier on users.  What you may not know if you haven't looked at EVOQ is that it's one of the main features of the commercial product.

For my users both keeping the website consistent and accessible has been a major chore, Occasionally, I get an extremely smart user who can do one of those two, but never both.   Now in the world of enterprise you strive to be both, but in the world of government it's an essential.  Lawsuits get brought and horrible press is written about sloppy websites or websites that ostracize a portion of the population like the blind or the physically limited. This is why I see Structured Content as a super important tool in the fight against user errors, it allows the users to essentially fill out a form with their data and then for me to control how that data is displayed.

Now many people will say the main benefit to this type of system is that you're creating a "Headless CMS" so you can redistribute the content through a myriad of interfaces like Alexa or Facebook messenger or name a end-point.  While this may seem like a great idea and your management may see all the great possibilities for this, the truth is you probably have enough trouble managing your development load already and real people don't want the "same" message in multiple platforms.  Think of it this way, just because you can get pizza delivered by car, boat, plane, bicycle, rickshaw, drone and para-glider does it mean that you want the EXACT same pizza delivered at the EXACT same time by ALL those methods? The answer is no, and if someone says "well most people only subscribe to us on one platform, not all of them", while that could be true for some it's probably not true for MOST.  If you like a brand you probably follow them on Twitter, Facebook, Instagram and check out their site from time to time.

So I'm looking at the Structured Content engine for having the control, not the capabilities.  That however, is not to say that I don't need capabilities in that control and there is the rub.  I need to be able to have all the control and commands I would with this flexible data as if i was writing an application to display it. More on that to come...

So what do you think? Do you have problems like this with users? How do you keep them from ruining the site? Does this sound like a path you would or will take?