The challenge was to produce a modern, re-skinnable, journal delivery platform. Our clients could migrate from our older Wiley Online Library offering to take advantage of a responsive design, more user friendly experience, and customizable content.
The project generated over $100m in revenue for Wiley & Sons Ltd, which helped in their purchase of Atypon - and hence the ultimate demise of the project. These notes aim to capture the essence of the beast that once was! By the end of the project, 25 websites had been produced.
What did I learn?
Some notable personal achievements were:
- To collaborate with our data services team and implement the search API, based on UX requirements.
- Integrate Avectra (a third party user authorization system) for a major publisher.
- Scrape and import thousands of inconsistently formatted paragraphs (Editor Highlights) to maintain publisher’s commentary on content, between Wiley Online Library versions.
- Configured SonarQube and Jenkins to build and evaluate projects to maintain code quality and provide feedback locally for developers.
- Working in a major corporation (over 5k employees), on a daily basis with teams ditributed across the globe (USA, UK, India, Russia, Germany).
- Participating in, and occasionally running the daily production monitoring meeting with key stakeholders from around the company.
I helped develop the product using the following technologies:
Our Team
Our team's responsibility was to deliver content between our datacentres and our frontend team, via dotCMS. We had to design systems based on incoming requirements, and collaborate with other teams to deliver production-ready code.
Working in an agile environment, on a global scale. Daily standups, technical analysis and task estimation, specification workshops, monthly restrospectives, iteration planning. I worked directly with both the frontend team and the dataservices team, to discuss requirements and solutions. I also raised any concerns with the UX team / Business Analysts, if requirements weren't obvious or missed particular scenarios. We used JIRA to tracked our work, and Slack enabled us to communicate remotely in real time, if needs be.
Wiley were good to us, we were allowed to take time to learn interesting new technologies in our many mini-hackathons. If it wasn't for the Atypon aquisition, and subsequent redundancies, I would have happily continued my career there.
Development
We developed locally in Windows 7 using a VM generated from our Ansible scripts. In Windows, we used the Eclipse IDE, Java 7, PuTTY with Pageant to SSH into our various environments. We used Gradle for building. We used Apiary both to mock up services and for our unit test suites.
Most of the code backend we developed was Java / Spring OSGI plugins which we used to extend the functionality of dotCMS. dotCMS provided a convenient way for our clients to be able to publish and edit their own content online.
Our workflow was fairly standard - develop locally in a VM, push changes to our git repos, which would automatically build and deploy the latest code and assets to our dev environment. Possibly do further development directly on the dev environment (for dotCMS push publishing reasons), then off to QA enviroment for functional testing. If all is good, on to staging for regression testing against real data and services, and then lastly to our performance testing environment before being approved for release. We would release every 3 months or so, with extra patch releases whenever necessary. Typically we would use Jenkins to build, deploy and publish to each environment, although some manual work was occasionally needed.
Towards the end of the project, the business was less keen to develop new features and so focus shifted towards increasing stability, improving documentation, and ironing out defects.
Prototype Webpages
Each website had 4 different types of webpages - a home page, a journal home page, search results page, and a blank page for any additional needs.
The idea being that each type of page could largley be populated based on convention - i.e. site-specific configuration and content authored in dotCMS. This approach made it easy and fast to produce new websites, albiet at the expense of a slight reduction in flexibility.
Journal Home Page
(Current Issue, Journal Highlights, Browser Articles, Resources, Etc.).
Perhaps one of the most significant OSGI plugins we developed was an AJAX friendly search results page. This allows users to search for articles within a publisher's catalogue of journals:
Search Results Page
Users could filter, sort, and paginate through results.
Probably one of the most challenging aspects of making the search work, was the UX requirements was had to follow. Although you can't see it here, the URL/s and subject taxonomies had been dictated by our clients and UX designers. Some of the requirments were impossible and some made our life downright difficult, but we managed to find a compromise by introducing some undesirable character conversions at various stages throughout the request / response cycle. Ultimately, our clients were happy and we ended up with more user-friendly URLs.
Conclusion
I enjoyed working with the team at Wiley. WOL2 was an interesting project to work on and the company encouraged innovation with annual hackathons and knowledge sharing activities. It gave me experience working with CMS's, which in turn has peaked my interest in serverless architectures, and I feel like I understand a lot more of the devops side of things now.