Archived Blackboard Learn discussions

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Thu, 10 May 2012 14:06:04 GMT

Good morning,

Ahhh - your at bundle 22 or higher and the SAIP web service target you've defined is set to version 2.0. If you were to create another target and set it's level at 1.5 - add bulkmanagementservice as mentioned above (then generate routings and go enable them) - and use your soapui to call that targetid you'll get the older stuff which I'm stuck working with and uses the transform as I outlined above.

If your'e stuck with 2.0 and want to change the output from the LIS version 2.0 target - I believe we followed the flow far enough to find that there were xsl transforms somewhere but that was so 29 days ago :)

For us, at this stage even after we upgrade to bundle22 I will recommend that we don't go to LIS 2.0 until we needed real-time integration and until the LIS integration catches up to Enterprise 1.1 in the mappings it supports.

I hope that helps.

Tim

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Zubair Ali — Thu, 10 May 2012 11:37:44 GMT

Thanks Tim,

@University Of Berlin

Actually i already build custom APIs in PLSQL and its running fine in production, but now client want to have SAIP to overcome programmer requirement.

@Tim,

My email is zubair648@gmail.com, if you feel to join me throug email then i have no problem.

I have seen your peoplecode of BULKDATAEXCHANGEMGMTSERVICE. I want to transform LISv1.0 files into Enterprise 1.1, currently i have configured follwoing operations on "Request Handler screen"

READGROUPSMGMT

READPERSONS

READCOURSETEMPLATES

READCOURSEOFFERINGS

READCOURSESECTIONS

READSECTIONASSOCIATIONS

READMEMBERSHIPS

these operations belongs to four web services like,

COURSEMANAGEMENTSERVICE

MEMBERSHIPMANAGEMENTSERVICE

GROUPMANAGEMENTSERVICE

PERSONMANAGEMENTSERVICE

currently configured SAIP module (Peoplesfot 9.0 bundle#15) is generating 7 files accrding to each operation on my FTP location.

I am invoking each operation in SoapUI.

NOw, i want to change each XML file to Enterprise v1.1, for this task, which application pacakges i have to change? (i can see at handler transformation page). how to invoke web services through script is also needed. Kindly also provide the detail/code which i have to raplace in each class.

Regards

Zubair Ali

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Zubair Ali — Thu, 10 May 2012 11:37:39 GMT

Thanks Tim,

@University Of Berlin

Actually i already build custom APIs in PLSQL and its running fine in production, but now client want to have SAIP to overcome programmer requirement.

@Tim,

My email is zubair648@gmail.com, if you feel to join me throug email then i have no problem.

I have seen your peoplecode of BULKDATAEXCHANGEMGMTSERVICE. I want to transform LISv1.0 files into Enterprise 1.1, currently i have configured follwoing operations on "Request Handler screen"

READGROUPSMGMT

READPERSONS

READCOURSETEMPLATES

READCOURSEOFFERINGS

READCOURSESECTIONS

READSECTIONASSOCIATIONS

READMEMBERSHIPS

these operations belongs to four web services like,

COURSEMANAGEMENTSERVICE

MEMBERSHIPMANAGEMENTSERVICE

GROUPMANAGEMENTSERVICE

PERSONMANAGEMENTSERVICE

currently configured SAIP module (Peoplesfot 9.0 bundle#15) is generating 7 files accrding to each operation on my FTP location.

I am invoking each operation in SoapUI.

Regards

Zubair Ali

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Wed, 09 May 2012 19:18:21 GMT

Your question is interesting.

What I've done (I believe) is the path of least customization within Peoplesoft and no customization in BB. With that said - If I had to invent this process would I do it this way? Not a chance.... From a process perspective however, I feel it fits. Because PeopleSoft was producing IMS (just not compatible) the production of groups, people, memberships and consumption by BB using a sync process doesn't change. Other than some tweeks to the dataset which I'm sure many institutes would need to do - The richness of the SAIP processes to define and manage LMS targets, academic orgs and course scope is worth it.

The PeopleSoft SAIP product has changed considerably between bundle 15 and 22 however - and in our brief look at the technology that SAIP has in place to support LIS - even Oracle (PeopleSoft) have gone away from this 'bash out XML' approach and are now using xsl transforms to produce LIS v2.0 output from internal structures. It's been awhile since we poked at bundle 22 and we didn't poke hard as it's going to be awhile till we get there.

Tim

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Hashemite University — Wed, 09 May 2012 15:23:21 GMT

HI Tim

Do you believe that this way is the right way, converting Standards programmatically , what you got from that? what is the special thing in IMS 1.1?, instead you could export BB data in a feed files easily, i can't get a point of doing such things

Regards

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Wed, 09 May 2012 13:36:03 GMT

your welcome - as far as the php goes - the php isn't checking anything - it waits passively to be involked by the peoplesoft bulkmanagementservice then takes action by posting the xml to blackboard.

As far as the changes to output Enterprise 1.1 - I tried reaching out to you directly by email. I'm not sure we want to litter this administrators forum with this - if we came up with something it would probably be better to move to edugarage.

As a teaser - you don't need to worry about the SA_SNAP_AE package - it is the overall app engine which performs the snapshot process.

take a peek at:

Set Up SACR -> System Administration -> Integrations -> Transform Handlers

if you brought up service PERSONMANAGEMENTSERVICE service operation READPERSONS - notice the Package Name and Application Class This is the gold which is producing all of the output which SA_SNAP_AE produces.

I created a new Application Package and created App classes with the same name - so for instance - for PersonTransformManager my customized (brutal code - remember I'm not a programmer) transofrm is attached.

I don't know that this will survive the cut and paste - but if you compare this to the original you'll see that this isn't a terribly tough change - mostly it's changing tag names and XML structure and I've added one piece of content which wasn't there originally.

Tim

--------------------------------------- cut here --------------------------------------

/* Tim - Macewan

   Changes to switch from whatever PeopleSoft spits out to a form which Blackboard 
   accepts called IMS Enterprise 1.1.  This was more or less hand crafted thru trial and error.  
   God bless SoapUI, XML Copy Editor and errorlogs...
*/

import SAE_FRAMEWORK:TRANSFORM:AbstractTransformManager;

class PersonTransformManager extends SAE_FRAMEWORK:TRANSFORM:AbstractTransformManager
   method transformRowset(&rowset As Rowset) Returns XmlDoc;
end-class;

method transformRowset
   /+ &rowset as Rowset +/
   /+ Returns XmlDoc +/
   /+ Extends/implements SAE_FRAMEWORK:TRANSFORM:AbstractTransformManager.transformRowset +/
   
   Local XmlDoc &inXMLDoc;
   Local XmlNode &personNode;
   Local XmlNode &level1Node;
   Local XmlNode &level2Node;
   Local XmlNode &level3Node;
   Local string &xmlString;
   Local number &start;
   Local number &len;
   Local Record &record;
   Local string &fullName;
   Local string &gender;
   Local string &recStatus;
   
   &inXMLDoc = CreateXmlDoc();
   
   /*
   We're going to try using this to figure out recstatus attribute...
*/
   If None(&rowset(1).SAE_PERSON_VW.SAE_MODE.Value) Then
      &rowset(1).SAE_PERSON_VW.SAE_MODE.Value = %This.objConstants.CHANGED;
   End-If;
   
   /*
   This is left over from where we stole this code - delete it when we know this is not required.
   %This.objTransformUtil.addExtension(&personNode, &rowset(1).SAE_PERSON_VW.SAE_MODE.Value);
   */
   
   /* this is BAD - I'm not sure what all the values are - so we're only going to deal with C for the moment
      any other values are going to leak thru and we'll need to catch them up...
      BAD BAD MAGIC - don't let this get to production :)
   */
   
   &recStatus = &rowset(1).SAE_PERSON_VW.SAE_MODE.Value;
   
   Evaluate &rowset(1).SAE_PERSON_VW.SAE_MODE.Value
   When "C"
      &recStatus = "2"
   End-Evaluate;
   
   &personNode = &inXMLDoc.CreateDocumentElement("person");
   &personNode.AddAttribute("recstatus", &recStatus);
   
   /* Source id tag*/
   &level1Node = &personNode.AddElement("sourcedid");
   &level2Node = &level1Node.AddElement("source");
   &level2Node.NodeValue = %DbName | "-" | %This.dataSource;
   
   Local string &emplid = &rowset(1).SAE_PERSON_VW.EMPLID.Value;
   &level2Node = &level1Node.AddElement("id").AddText(&emplid);
   
   /* Name tag*/
   &level1Node = &personNode.AddElement("name");
   &record = &rowset(1).GetRowset(Scroll.SAE_F_NAMES_VW)(1).SAE_F_NAMES_VW;
   &level2Node = &level1Node.AddElement("fn").AddText(&record.NAME_DISPLAY.Value);
   &level2Node = &level1Node.AddElement("n");
   &record = &rowset(1).GetRowset(Scroll.SAE_NAMES_VW)(1).SAE_NAMES_VW;
   &level3Node = &level2Node.AddElement("family").AddText(&record.LAST_NAME.Value);
   &level3Node = &level2Node.AddElement("given").AddText(&record.FIRST_NAME.Value);
   
   /* probably a long debate whether we should send address at all... */
   /* - I will play with it for now */
   /* - extadd doesn't seem to map to anything in Blackboard */
   /* - it looks like we'll loose Address.3 and 4 due to lack of fields to map to... */
   /* - one idea might be to give postal, city, province, country but no street address...*/
   /* Address*/
   &level1Node = &personNode.AddElement("adr");
   &record = &rowset(1).GetRowset(Scroll.SAE_ADDRESS_VW)(1).SAE_ADDRESS_VW;
   &level2Node = &level1Node.AddElement("extadd");
   &level2Node = &level1Node.AddElement("street").AddText(&record.ADDRESS1.Value);
   &level2Node = &level1Node.AddElement("street").AddText(&record.ADDRESS2.Value);
   &level2Node = &level1Node.AddElement("locality").AddText(&record.CITY.Value);
   &level2Node = &level1Node.AddElement("region").AddText(&record.STATE.Value);
   &level2Node = &level1Node.AddElement("country").AddText(&record.COUNTRY.Value);
   &level2Node = &level1Node.AddElement("pcode").AddText(&record.POSTAL.Value);
   
   /* no love with Telephone so far the tel tag doesn't seem to map to BB home telephone */
   /*Contact info*/

   /* userid goes here as well - this is way different than the original transform... */
   
   Local Rowset &wrkoprRec = CreateRowset(Record.PSOPRDEFN);
   Local number &i = &wrkoprRec.Fill("where EMPLID = :1", &emplid);
   If &i > 0 Then
      &level1Node = &personNode.AddElement("userid").AddText(&wrkoprRec(1).PSOPRDEFN.OPRID.Value);
   Else
      &level1Node = &personNode.AddElement("userid");
   End-If;
   
   &record = &rowset(1).GetRowset(Scroll.SAE_EMAILADD_VW)(1).SAE_EMAILADD_VW;
   &level1Node = &personNode.AddElement("email").AddText(&record.EMAIL_ADDR.Value);
   &record = &rowset(1).GetRowset(Scroll.SAE_PERPHONE_VW)(1).SAE_PERPHONE_VW;
   &level1Node = &personNode.AddElement("tel").AddText(&record.PHONE.Value);
   
   /*Demographics*/
   &level1Node = &personNode.AddElement("demographics");
   Evaluate &rowset(1).GetRowset(Scroll.SAE_PERSDATA_VW)(1).SAE_PERSDATA_VW.SEX.Value
   When "U"
      &gender = "0"
   When "F"
      &gender = "1"
   When "M"
      &gender = "2"
   End-Evaluate;
   &level2Node = &level1Node.AddElement("gender").AddText(&gender);
   &level1Node = &level1Node.AddElement("bday").AddText(&rowset(1).SAE_PERSON_VW.BIRTHDATE.Value);
   
   /*Datasource*/
   &level1Node = &personNode.AddElement("datasource");
   &level1Node.NodeValue = %DbName | "-" | %This.dataSource;
   /*
   If None(&rowset(1).SAE_PERSON_VW.SAE_MODE.Value) Then
      &rowset(1).SAE_PERSON_VW.SAE_MODE.Value = %This.objConstants.CHANGED;
   End-If;
   %This.objTransformUtil.addExtension(&personNode, &rowset(1).SAE_PERSON_VW.SAE_MODE.Value);
*/
   
   Return &inXMLDoc;
end-method;

--------------------------------------- cut here --------------------------------------

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Zubair Ali — Wed, 09 May 2012 12:46:33 GMT

Thanks TIM,

I got your point, i noted your PHP file and i am assuming that you placed this file on PHP server and scheduled it to check and invoke every time BB endpoint to handover the Snapshot file.

Also i am still confused how you transform LIS1.0 messages/XML to Enterprise 1.1, like i renamed application package SAE_FRAMEWORK, particularly SAE_SNAP_AE application engine, then how i would transform or shift the coding to Enterprise 1.1.

Regards

Zubair Ali

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Wed, 09 May 2012 12:09:59 GMT

Hi,

I'm not much of a programmer - I mostly credit Google for this code...

In my case I called this bulktransfer.php - and in my PeopleSoft SAIP navigate to:

Set Up SACR -> Product Related -> SA Integration Pack -> Set Up Targets -> Web Service Targets

for your blackboard server, under the Integration Metadata Settings, add the following service operation (if it doesn't already exist) and setup as follows:

Service: BULKDATAEXCHANGEMGMTSERVICE

URL: http:<***your php server***>/bulktransfer.php

This is a bit trimmed down version of my code - I'm only providing the portion of this which takes the file from PeopleSoft and posts it to Blackboard. I initially used this along with SOAPUI to call the PeopleSoft service operations for readgroups, readpersons etc.

good luck!

tim

--------------------------------------------- cut here -------------------------------------------------

<?php

      $xml = file_get_contents('php://input');

      $doc = new DOMDocument();
      $doc--->preserveWhiteSpace = false;
      $doc->loadXML( $xml );

      $operation = $doc->getElementsByTagName('operationName')->item(0)->nodeValue;
      $file = substr(strrchr($doc->getElementsByTagName('url')->item(0)->nodeValue,"/"),1);
      $savepoint = $doc->getElementsByTagName('savePoint')->item(0)->nodeValue;

      // these next 4 items are environment specific...
      // the first 3 come from the integration you create in Blackboard
      // the last item ($dir) is a cheat - This PHP server is the same as the FTP server which PeopleSoft is sending integration files to.
      // once PeopleSoft put's the file - I just open it and read it into a string (also evil as one day I'm sure this will blow up over
      // a memory issue if this doesn't get replaced before production.

      $URL = "http://<***your blackboard node name***>/webapps/bb-data-integration-ims-xml-<***your blackboard environment name***>/endpoint";
      $USER= "<***integration generated username which looks like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx***>";
      $PASS= "<***password***>";
      $dir= "/mount/disk1/ftp/res/home";

      $handle = fopen("$dir/$file", "r");
      $XMLPost = fread($handle, filesize("$dir/$file"));
      fclose($handle);

      $ch = curl_init($URL);
      curl_setopt($ch, CURLOPT_MUTE, 1);
      curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
      curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
      curl_setopt($ch, CURLOPT_POST, 1);
      curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
      curl_setopt($ch, CURLOPT_USERPWD, $USER.":".$PASS);
      curl_setopt($ch, CURLOPT_POSTFIELDS, $XMLPost);
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
      $output = curl_exec($ch);
      curl_close($ch);

      session_write_close();

--------------------------------------------- cut here -------------------------------------------------

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Wed, 09 May 2012 11:28:14 GMT

Hi there - actually according to Blackboard themselves it's supported.

In an earlier post Micheal referenced a new document:

It looks like Blackboard has just published an SIS Data Integration Handbook, which is linked at the bottom of KB article on the snapshot client being deprecated (https://blackboard.secure.force.com/btbb_TSGArticleView?Id=50170000000WmqnAAC&RecordType=01270000000Hip5AAC). The handbook is a Word document, which may mean that it's still a work in progress, but the direct link is https://blackboard.secure.force.com/AttachmentRedirect?bname=Kb-Attachments&fname=SIS+Integration+Framework+handbook-2012-4-30-15-46-50.docx&aws=AKIAJ7CS4QGS2DUC7WIA&sec=9kj8vt7mrLqMEn6Tfx3NCB/jWJkawjKmoUcXpwAS&uval=article

Mike

at the beginning it states:

The LIS 2.0 integration type provided out of the box starting with Learn 9.1SP6 is perhaps the easiest to setup and meets the basic needs for populating Learn with the essential data for operation.

Data supported: Users, Courses, Memberships (Enrollments, Staff assignments), and Terms [SP8+]

Requirements:

Learn SP6+

Peoplesoft versions:

Peoplesoft Campus Solutions 9, bundle 22 or later;

we're not at bundle 22 so we're sticking with IMS 1.1 Enterprise. From what I can see - having played with bundle 22 in a sandbox environment - my tactic of creating a separate app package to hold my custom transforms will survive the upgrade and require minimal (if any) re-work..

One thing to be cautious with when considering LIS 2.0 integration is whether it supports all of the data your expecting to integrate.... I recently saw a presentation slide during a Blackboard upgrade cohort session on SIS integration that listed which integration supported which types. Nodes (important to institutional hierarchy in community) isn't supported within LIS 2.0 (yet) and I also believe that Categories are not supported via ANY of the IMS integration types.

We're just in the process of moving to Blackboard 9.1 - so it's a double whammy of learning what the new LMS will offer us - how much of that functionality can be supported via IMS integration - and how much of that can be provided via SAIP.

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Zubair Ali — Wed, 09 May 2012 08:22:51 GMT

Hi ,

Wow, Wow, i am glad to hear that someone has same environement as mine, thanks @Tim Crisall , @Michaell and @University Of Berlin for your advices and help.

Now I need more help from you guys, specially from Tim, hope Tim you will not mind to give up some valuable time.

I need to learn all this process as you mentioned hear in the following lines:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------

It was a PeopleSoft development effort to make a new application package, to copy all of the delivered transforms into it, and then customize the transforms to output IMS enterprise XML.

Using this technique I've been able to create 'custom transforms' which I enabled thru PeopleSoft configuration and the output is IMS Enterprise 1.1 XML for group (sections, terms, hierarchy nodes and associated relationships), person (students and instructors) and membership (enrollment).

------------------------------------------------------------------------------------------------------------------------------------------------------------------------

It will be very appriciable if you share Code of Application Designer and PHP with me, i will be very helpful for me.

Thanks

Zubair Ali

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Hashemite University — Wed, 09 May 2012 07:52:02 GMT

Thank you Tim for sharing your knowledge with us, "At bundle 22 SAIP is capable of outputting LIS v2.0 web services and I have successfully played with getting person, classes and membership to message between a bundle 22 People Soft and BB Learn 9.1 SP8."

So can we say that there is a "live Integration" between PS bundle 22 and BB SP8, using the new standard LIS 2.0, using SOAP messages

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Michael Zimmerman — Tue, 08 May 2012 17:27:36 GMT

Thanks for sharing all your hands-on experience with SAIP, Tim!

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Tim Crisall — Tue, 08 May 2012 17:24:23 GMT

Good morning,

I just came across this post and I thought I could share some of my self-taught experience integrating PeopleSoft Campus Solutions 9.0 using SAIP with BB Learn 9.1 SP8.

First off - it's important to understand a bit of the lineage of the SAIP product. It was originally introduced as a bolt-on product - merged with the base Campus Solutions product in bundle 10, then had significant functionality enhancements at CS bundle 12, 15 and 22 (there may be others but that’s close). I found it necessary to do this research in order to get an accurate (although slim) set of documentation for my PeopleSoft release.

Up to bundle 22 the IMS which SAIP output was IMS Enterprise Services v1 - which is not at all compatible with the IMS Enterprise v1.1 snapshot XML or LIS v2.0 web services which BB Learn 9.1 SP8 is capable of. At bundle 22 SAIP is capable of outputting LIS v2.0 web services and I have successfully played with getting person, classes and membership to message between a bundle 22 PeopleSoft and BB Learn 9.1 SP8.

In our particular case we are behind in our PeopleSoft Patching - and are currently in production at Bundle 15. This means that the XML output of SAIP is not understood by ANY of the BB IMS integrations.

What I've done to overcome this - is to replace the delivered PeopleSoft transforms provided as part of SAIP. Fortunately - this is made quite easy to do. It seems that PeopleSoft SAIP was written to fit into an overall framework within Campus Solutions called the SAE framework. This framework is leveraged to produce the event triggers required for real time integration or to flag transactions to later be picked up during batch based sync operations. As mentioned, using Setup SACR screens any of the individual service operations can be assigned a transform other than what was delivered.

It was a PeopleSoft development effort to make a new application package, to copy all of the delivered transforms into it, and then customize the transforms to output IMS enterprise XML.

Now - as I'm still at the proof of concept stage, I've built a PHP 'engine' to play the role of batch engine to orchestrate the integration. This should be replaced and moved back into PeopleSoft before we hit production but this is currently not developed.

The PHP code sends a SOAP message to PeopleSoft Integration broker kicking off the readgroups service operation within the groupmanagementservice. This SOAP call returns a 'promise' that PeopleSoft will fulfill the job and a FTP URL where you can expect the results to be 'when the promise is fulfilled' (in reality this is when a PeopleSoft AppEngine completes).

When configuring my SAIP target system I enabled a service operation called BULKDATAEXCHANGEMGMTSERVICE which will send a SOAP operation to a defined URL whenever a SAIP sync operation is completed. In this case - I have the operation sending a soap message back to my PHP code which now knows the 'promise' is fulfilled. The PHP then takes the contents of the file from the FTP server and POSTS this to the Blackboard Learn IMS 1.1 integration web service Endpoint URL along with a BasicAuth header using the username and password setup when the BB integration was defined.

This cycle is repeated within the PHP for readCourseSections, then readPersons and last readMemberships.

It’s unfortunate that all of the shiny brochures for all of these products make it sound ‘easy’ and the fact that we use ‘standards’ is great. The devil is always in the details and you really have to get under the hood to first understand it – then to make it work. I’m feeling very optimistic about our ability to use SAIP within our University to integrate with Blackboard Learn 9.1 SP8.

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Michael Zimmerman — Wed, 02 May 2012 16:19:09 GMT

Carmine,

The data integration framework is primarily about putting SIS info into Blackboard. It's possible that SAIP in PeopleSoft may allow you to use the LIS integration to pull some information back from Blackboard, but I really don't know--you'd probably want to talk with your PeopleSoft folks about that.

Mike

Re: we need to integrate Peoplesoft Campus Solution with Black Board

Carmine Casciato — Tue, 01 May 2012 17:19:21 GMT

Thanks Berlin, just wanted to verify.