Cloud Chronicles: (Part 3 of 3) Hybrid Cloud & Enterprise Application Integration (EAI) Patterns & Practices
Ryan Johnson
Senior Engineering Manager (Fullstack) | Cloud & Data Architect | Leader in Digital Innovation & Analytics
This article is structured into three sections, investigating the fusion of on-premises data centers with cloud services, hybrid cloud integration patterns, reflections from the author, and key takeaways and lessons learned from the underlying project. The initial sections delve into integration principles, the influence of human nature on design, security measures for on-premises data centers, and the establishment of secure connections into the cloud. Parts 1 and 2 particularly concentrate on crucial considerations for Enterprise Application Integration (EAI) solutions, offering guidance for IT professionals and addressing common challenges, especially for cloud and solution architects navigating integration complexities stemming from hybrid hosting scenarios. The article attempts to provide best practices for leveraging (both brokered real-time and batched) messaging / event processing scenarios; highlighting the ongoing importance of continuous testing, documentation, and adaptability to evolving security landscapes. These sections lay the groundwork for 2 subsequent discussions from a post-mortem perspective---concluding with Part 3 that presents insights / lessons derived from the project forming the basis of this discussion (covering aspects such as layered security, verifiability, observability, durability, repeatability, and performance and extending to meeting requirements related to availability, latency, operations per second, throughput, resiliency, authentication and authorization, telemetry, logging, and recoverability).
Analyzing the Lessons Learned
Technology
???????? Integrating an on-prem SharePoint farm & Salesforce will require middleware for extensive & poly messaging.?
???????? SharePoint Timer Jobs may be scaled but for near real-time processing, but Windows & Azure Data services are the more ideal on-prem. solution.
???????? For continuous crawl & indexing on very large & volatile data sources, a dedicated search farm with stand-alone servers for crawl, index, & query operations if rapid scaling is necessary (as opposed to the more economical approach of using clustered shared service instances)
???????? Automated SharePoint Site Provisioning is difficult to industrialize, but with pre-optimized site templates that auto hydrate permission seeding can support scale provisioning with moderate latencies (we observed site provisioning rates of around 500 new site collections in under 15 minutes—depending on the rate of generation / throughput for net-new opportunities in the Salesforce parent site)
???????? If your client requires SharePoint and Salesforce to conjoin as a robust document repository & workflow engine, avoid real-time integrations for defining SharePoint site hierarchies and avoid breaking permission inheritance models that require unique ACLs (access control lists) containing granular user-permissions (as opposed to preconfigured security groups using predefined role-based-access-controls –i.e., discrete authorized verb-based operations)
EAI Selection Criteria
???????? Real time integration/ESB - BizTalk Server/ESB Relays
???????? Scheduled/Bulk processing/(From/To) database – SSIS.
???????? Batch applications should leverage QUEUES for brokered messaging.?
???????? On-demand low latency services – WCF/SOAP/REST/Web Sockets
o REST is best for stateless CRUD operations and HTTP verbs match operation o Orchestration still needs WCF/SOAP o WebSocket are excellent for bi-directional compound operations that have multiple step-up quality gates & record state requirements before additional sequential operations
Middleware Messaging
???????? Real-time integration should be avoided whenever possible.
???????? Decouple, decouple, decouple.
???????? Transactions that can cause race conditions and deadlock needs to be thoroughly understood and accounted for
???????? Stateless is not always possible, sessions need monitoring and cleanup.?
???????? REST is not always best, but SOAP service contracts are tightly coupled, choose wisely.
???????? SSIS is still a viable solution for data integration.
???????? Azure Messaging can be used by on-prem and cloud-hosted applications (via Queues, Topics, & Relays)
o With Azure Relays, each connection is established inside the datacenter, so firewalls allow incoming traffic to each application without opening new ports
???????? CORS will continue to be a problem for RESTful calls from client browsers.
???????? Queues enable applications to communicate even when both aren't running, handy with batch and mobile applications.
???????? Relays are the right solution when you need direct communication between applications.
???????? Respect ACID principles (Atomicity, Consistency, Isolation, Durability)
???????? SOLID (single responsibility, open-closed, Liskov substitution, interface segregation and dependency inversion)
?
Our Approach
Approaching the integration of SharePoint with Salesforce involves a series of strategic and technical steps. Here's how we approached the tasks of integrating the two platforms:
Needs Assessment and Goal Definition
Identify the Need: Understand why you're integrating SharePoint with Salesforce. Is it for better document management, enhanced collaboration, or improved data accessibility?
Define Objectives: Clearly outline what you want to achieve with the integration. This could be streamlined operations, enhanced user experience, or reduced manual data entry.
Stakeholder Engagement
Involve Stakeholders: Engage with all parties affected by the integration, including IT, sales, customer service, and management. Gather their input and understand their needs.
Define Roles and Responsibilities: Establish who will be responsible for each part of the integration process, from planning to execution and maintenance.
Technical Assessment
Evaluate Current Systems: Assess the current state of your Salesforce and SharePoint environments. Understand their configurations, customizations, and how they are currently used.
Identify Technical Requirements: Determine what technical components you'll need for the integration, such as APIs, middleware, or third-party integration tools.
Integration Planning
Choose an Integration Method: Decide whether to use a standard integration tool, custom development, or a combination. Tools like Salesforce Connect or third-party middleware might be considered.
Map Data and Processes: Understand how data flows between systems and how documents are used in business processes. Determine which data needs to be synchronized and how often.
Security and Compliance
Assess Security Requirements: Ensure the integration adheres to data protection regulations and both platforms' security guidelines.
Plan for Compliance: Make sure that the integration will not compromise any compliance standards relevant to your industry or data types.
?
Implementation
Develop the Integration: Based on the chosen method, develop the connection between Salesforce and SharePoint. This might involve configuring Salesforce Connect, writing custom code, or setting up a third-party tool.
Integrating Salesforce CRM with on-premises SharePoint Document Management System (DMS) can significantly enhance business operations by streamlining data flow and improving document accessibility. This article provides a high-level guide to help architects tasked with integrating hybrid cloud hosted appliations.
领英推荐
Step 1: Planning and Strategy
Identify Objectives: Clearly define what you want to achieve with the integration.
Map Data and Processes: Understand how data flows between systems and how documents are used in business processes.
Security and Compliance: Determine the security measures and compliance requirements.
Step 2: Setting up the Environment
Install Required Tools: Tools like Salesforce Connect and SharePoint REST API might be needed.
Network Configuration: Configure firewalls and other network components to allow secure communication.
Step 3: Developing the Integration
Create External Data Sources in Salesforce: This allows Salesforce to read data from SharePoint.
Use Salesforce Connect: To map external objects in Salesforce to corresponding documents in SharePoint.
Develop Custom Solutions: If necessary, develop custom solutions using Salesforce Apex and SharePoint APIs.
Step 4: Data Mapping and Synchronization
Map Fields: Ensure that fields between Salesforce and SharePoint are properly mapped.
Synchronization: Decide on the synchronization frequency and direction (bidirectional or unidirectional).
Step 5: Testing
Unit Testing: Test individual components of the integration.
System Testing: Test the complete integration in a controlled environment.
User Acceptance Testing (UAT): Let a select group of end-users test the integration.
Step 6: Deployment and Training
Deploy Gradually: Consider a phased deployment to monitor performance and gather feedback.
Training: Provide comprehensive training to end-users and administrators.
Step 7: Maintenance and Support
Monitor: Regularly monitor the integration for any issues.
Updates: Keep both Salesforce and SharePoint updated to the latest versions.
Support: Establish a support protocol for any issues that users encounter.
Best Practices
Regular Backups: Regularly back up data from both systems.
Documentation: Maintain detailed documentation of the integration process and architecture.
Security Reviews: Regularly review security settings and access controls.
Integrating Salesforce CRM with an on-premises SharePoint Document Management System can transform how your organization manages and accesses critical customer data and documents. By following the steps outlined in this guide, businesses can ensure a smooth integration process that maximizes productivity and enhances collaboration. As with any integration, it's important to continually monitor and adjust strategies to align with evolving business needs and technology advancements.
SharePoint Auto-Provisioning
Creating and managing team sites efficiently in SharePoint is crucial for organizations looking to streamline their collaboration processes. Auto-provisioning team sites using timer jobs is a sophisticated method that automates the creation and setup of these sites based on specified schedules and criteria. This article will guide you through the process of setting up auto-provisioning for SharePoint team sites using timer jobs.
Timer Jobs
Timer jobs in SharePoint are scheduled tasks executed by the SharePoint Timer Service. These tasks perform maintenance activities such as cleaning databases, sending email alerts, and, importantly for our purposes, auto-provisioning team sites. They are essential for automating repetitive tasks, ensuring that your SharePoint environment is dynamic and efficiently managed.
Prerequisites
-??????????? SharePoint Admin Access: You need to have administrative privileges on your SharePoint environment to create and manage timer jobs.
-??????????? Development Environment: Familiarity with Microsoft Visual Studio is required as you'll be developing a custom SharePoint solution.
-??????????? Basic Knowledge of C#: Timer jobs are generally written in C#, so understanding this language is necessary.
Development Approach
-??????????? Install Visual Studio: Ensure you have Microsoft Visual Studio installed.
-??????????? Setup SharePoint Development Tools: Install necessary SharePoint development tools and SDKs in Visual Studio.
-??????????? Create a Project: In Visual Studio
-??????????? Add a Timer Job: Right-click the project → Add → New Item → Code → Timer Job.
-??????????? Write the Logic: In the newly created timer job file, write the C# code that defines what the timer job does. Here's a rough outline of tasks:
o?? Check for a list of team sites to be created (possibly from a database or list).
o?? For each site, use the SharePoint API to create the site with the specified template, permissions, and content.
-??????????? Error Handling: Ensure proper error handling and logging mechanisms are in place.
-??????????? Build the Solution: Compile the project in Visual Studio.
-??????????? Deploy: Deploy the compiled solution to your SharePoint environment.
-??????????? Activate the Feature: Timer jobs are typically deployed as features, so ensure the relevant feature is activated in your SharePoint environment.
-??????????? Test: Monitor the timer job's operation through the SharePoint Central Administration website and verify that team sites are being provisioned as expected.
Best Practices and Considerations
-??????????? Schedule Wisely: Consider the frequency of team site creation and set your timer job's schedule accordingly to minimize performance impacts.
-??????????? Security: Ensure the account running the timer job has the appropriate permissions to create sites.
-??????????? Logging and Monitoring: Implement comprehensive logging to troubleshoot and monitor the timer job.
-??????????? Scalability: If you anticipate a large number of sites being created, plan for scalability and performance.
Auto-provisioning SharePoint team sites using timer jobs is an efficient way to manage site creation and maintain an organized digital workspace. By following the steps outlined above, you can automate this process, saving time and ensuring consistency in how team sites are set up and configured. Remember to adhere to best practices and continuously monitor and update the timer job to adapt to any changes in your organizational structure or SharePoint environment. With the right setup, auto-provisioning will significantly enhance your team's collaboration and productivity
nbsp;
Approaching the integration of SharePoint with Salesforce involves a systematic and strategic process. Initially, assess the needs and define clear objectives for the integration, focusing on improved data management and collaboration. Engage stakeholders from various departments to gather input and assign responsibilities. Conduct a technical assessment of your current systems and identify the necessary components for integration. Choose an appropriate integration method, like using Salesforce Connect or custom development, and map out the data and processes involved. Prioritize security and compliance to protect data and adhere to regulations. Develop and thoroughly test the integration, ensuring it meets user needs and system requirements. Prepare comprehensive documentation and train users to facilitate a smooth transition. Deploy the integration gradually, continuously monitor its performance, and collect feedback for optimization. Regularly review and adjust the integration to ensure it remains effective and aligned with business objectives.
?