PeSIT

PeSIT^[1] (Protocol d’Echanges pour un Systeme Interbancaire de Telecompensation) is a file transfer protocol developed in 1986 by the French Interbank Teleclearing System Economic Interest Grouping (GSIT). Designed by working groups of file transfer experts from large computing centers, it offers basic file transfer functionalities and advanced features for managing mass file transfers and organizing industrial-scale file transfer services. PeSIT connections occur between two named and identifiable partners. The initiator of a connection, acting as the PeSIT requester, negotiates with the PeSIT responder for sending or receiving a file, sending a message, or acknowledging a transfer. The negotiation for file transfer does not involve mentioning the file name or its location in the partners' file systems (FS) but rather a virtual file identified by an identifier, "filename," which corresponds to local sets of definitions for accessing and reading a file on the sender's side and creating and writing it on the receiver's side. Each transfer has a unique identification known by both exchange partners. The protocol allows assigning a priority to each transfer, which should be respected by both parties.        

File exchanges between computers predated the public opening of the Internet in 1990, with producers offering their proprietary networks, like IBM's SNA or DEC's DECNET, for connecting computers from the same manufacturer. The breakthrough came with public X25 networks in 1976, allowing interconnectivity regardless of equipment origin. The French X25 network operator, Transpac, started in 1978, offering banks a means to accelerate file exchanges between clients and banks (through the rudimentary ETEBAC3 protocol, defined by CFONB in 1984). GSIT, established in 1983, was tasked with organizing exchanges between banks. The first version of the PeSIT protocol linked adherents to the SIT-Interbank Compensation System network in 1992, initially defined for use over X25, X32, PAD, ISO 8802.3, and NETEX, and later extended to TCP/IP and APPN. The latest version E of PeSIT was published in September 1989 (look for at pesit.org) covering various profiles for use within and outside the SIT network for bank-client exchanges, as well as B2B and B2C exchanges. Significant regulatory changes in financial markets were introduced following the Lisbon European Council in 2000 and the euro introduction. The European Payments Council (EPC) was formed in June 2002 to establish the Single Euro Payments Area (SEPA), replacing the SIT network with the STET-CORE system in 2008 and dissolving GSIT in 2011. However, due to its efficient implementations, PeSIT remains widely used today for B2B and B2C exchanges not only in France but across Europe and beyond the EU. 

The latest version E of PeSIT, published in September 1989 (see pesit.org), defines four PeSIT profiles:

• PeSIT SIT
• PeSIT OFF SIT
• PeSIT OFF SIT Secure
• ETEBAC5.

- designed exclusively for the SIT network and is no longer in use.

- used for exchanges between banks and their clients, has become the most widespread profile for exchanges outside financial transactions and is supported by all products that announce PeSIT protocol support.

- included data security functionalities like DES and RSA encryption or file sealing, but was quickly surpassed.

- a derivative of PeSIT OFF SIT Secure with only RSA encryption, was chosen by CFONB as the secure PeSIT OFF SIT protocol for sensitive file exchanges between banks and large companies. Its complexity and limited implementations prior to the arrival of SEPA standards meant it haven't gained popularity.

The PeSIT protocol functions as an abstract machine where FPDU (File Transfer Protocol Data Unit) messages are exchanged between two homologous PeSIT units: the connection initiator (requester) and the server (responder). These messages contain a protocol header and a variable area with PIs (Parameter Identifiers) featuring protocol parameters or file data.

A PeSIT session unfolds in various phases, allowing partners to coordinate the execution of triggered services through a request FPDU (=>) and acknowledged by a response FPDU (<=). Services for disconnecting, interrupting a transfer, and resynchronization can be initiated by either the requester or the responder. The FPDU.ABORT is never acknowledged.

The process for each file transfer starts with its negotiation through FPDU.CREATE for sending and FPDU.SELECT for receiving. File names and their locations are determined by local PeSIT services. Local virtual file definitions on each side dictate access to the file and operations on records during reading or writing. File names and their locations are at the discretion of local PeSIT services. Local virtual file definitions on each side determine the way to access the file and the operations that must be performed on records during reading or writing. All necessary operations to prepare for sending and receiving files are coordinated through selection and opening services. Writing and reading services allow positioning within files in case of transfer resumption.

File data is sent record by record, with or without flow control and setting of resume points. Sending the FPDU.DTFEND by the sender triggers the end of data transmission service, which precedes the execution of the end of transfer service triggered by the requester through the sending of FPDU.TRANSEND. The parameters of FPDU.TRANSEND allow for the control of the number of records and bytes sent and received. The transfer's end is followed by the file closure service triggered by FPDU.CRF, then the deselection service (FPDU.DESELECT).

It is the requester who ends the connection phase by sending FPDU.RELEASE. Each response FPDU carries the diagnostic code of the service execution result.

• Connection Establishment Service
  • => FPDU.CONNECT,
  • <= FPDU.ACONNECT or FPDU.RCONNECT
• Connection Release Service:
  • => FPDU.RELEASE,
  • <= FPDU.ACK(RELEASE)
• Connection Cut-off Service:
  • => FPDU.ABORT

• File Creation Service:
  • => FPDU.CREATE,
  • <= FPDU.ACK(CREATE)
• File Selection Service:
  • => FPDU.SELECT,
  • <= FPDU.ACK(SELECT)
• File Deselection Service:
  • => FPDU.DESELECT,
  • <= FPDU.ACK(DESELECT)
• Message Transfer Service:
  • =>  FPDU.MSG (FPDU.MSGDM, FPDU.MSGMM, FPDU.MSGFM),
  • <= FPDU.ACK(MSG).

• File Opening Service:
  • => FPDU.ORF,
  • <= FPDU.ACK(ORF)
• File Closure Service:
  • => FPDU.CRF,
  • <= FPDU.ACK(CRF)

• File Writing Service:
  • => FPDU.WRITE,
  • <= FPDU.ACK(WRITE)
• File Reading Service:
  • => FPDU.READ,
  • <= FPDU.ACK(READ)
• Data Transmission Service:
  • => FPDU.DTF (FPDU.DTFDA, FPDU.DTFMA, FPDU.DTFFA)
• Synchronization Point Setting Service:
  • => FPDU.SYN,
  • <= FPDU.ACK(SYN)
• End of Data Transmission Service:
  • => FPDU.DTF_END
• End of Transfer Service:
  • => FPDU.TRANS_END,
  • <= FPDU.ACK(TRANS_END)
• Transfer Interruption Service:
  • => FPDU.IDT,
  • <= FPDU.ACK(IDT)
• Resynchronization Service:
  • => FPDU.RESYN,
  • <= FPDU.ACK(RESYN)

The phases of a PeSIT service session are nested and executed sequentially (two transfers cannot be executed in parallel during the same session. To exchange two files simultaneously two sessions have to be opened).

These are the protocol's capabilities that have solidly embedded PeSIT OFF-SIT implementations into the IT service infrastructures of large enterprises.

Connection establishment is initiated by the requester sending FPDU.CONNECT, which contains both its identification and the responder's identification, as well as the password. This allows the responder to identify and authenticate the requester, and also allows the requester to authenticate the responder. FPDU.CONNECT with the responder's identification enables the building of intelligent PeSIT proxies.

Upon connection establishment, both partners negotiate the use of synchronization point or resynchronization services.

The requester can send a message containing up to 4095 bytes. In addition to the end-of-transfer service, which coordinates the closing and deselection of files between two exchange partners, the file receiver can send a specific message known as the file receipt acknowledgment message.

The requester can send or request to receive a file by announcing its protocol characteristics, such as the virtual file name (PI12) - the virtual file identification known and defined by both partners, file size to be transferred (PI41 and PI42), file organization (PI33) which can be sequential, relative, or indexed, record format (PI32) which can be fixed or variable, record length (PI31). Key position (PI38) and key length (PI39) are notified in the case of an indexed file. Data code (PI16) can be ASCII, EBCDIC, or BINARY. The use of compression (PI21) is negotiated between partners.

Each file or message transfer has an identification assigned by the sender and notified to the receiver, allowing for tracking and control of each transfer from end to end.

Each transfer must have a priority level assigned by the transfer requester, which is notified to the responder.

Both the negotiation of a file transfer and the process of ending a transfer involve the execution of services that allow control of their outcome and notification of any potential failures at the same stage of the respective phases.

The protocol allows for interrupting (or suspending) an ongoing transfer by sending FPDU.IDT. Interrupted transfers, whether voluntary or not, can be resumed if they are executed with flow control. During such a transfer, control points (checkpoints) must be set, which will be used to reposition in both files at the location corresponding to the exchanged and recorded data when sending FPDU.ACK(SYN).

Each transfer can be made "on behalf of" and "to the destination of" thanks to Customer Id (PI61) and Bank Id (PI62). These represent the origin and final destination of the file transmitted between the requester and responder of the current transfer stage. The file receiver must know the adjacent partner knowing the path leading to the recipient. The mechanism of sending a file receipt acknowledgment message allows the recipient to notify the good reception of the file. In the case of Store and Forward type sending, the acknowledgment message makes its way back through all the intermediary points crossed during the file's sending.

All service FPDU, including connection establishment, connection release, file creation, file selection, and file deselection, can contain a user parameter of 254 bytes (PI99). Two FPDU from the selection phase; FPDU.CREATE and FPDU.ACK(SELECT), contain a free user parameter of 80 bytes (PI37). These two parameters provide the ability to pass information to user applications and even customize the operation of PeSIT services.

In comparison with the two most well-known protocols: FTP/FTPS^[2] and SFTP,^[3] PeSIT OFF-SIT offers advantages presented in the table below.

File transfer and IT services are the most resource-intensive in terms of human and material resources. Orchestrating and maintaining production pace in companies where the number of files exchanged is constantly growing requires expertise in data storage, systems, networking, security, and the use of products and standards. On the network side, the use of bandwidth by file transfer flows was highly detrimental to interactive exchanges. The technological environment of the 80s and 90s demanded much stricter rigor in choosing the architecture of IT infrastructures and services.

Designed by experts with years of experience in managing massive file transfer services daily in banking institutions, the PeSIT protocol was the most refined and best suited to meet all these needs and constraints.

The first products supporting the PeSIT protocol, outside the SIT framework (as well as ETEBAC5), appeared in the 1980s. At that time, there was more talk of file transfers monitors than of MFT^[4] (Managed File Transfer). The difference between these two types of products lies more in the mode and level of interaction with enterprise IT systems than in the means of control and tracking of transfers. Transfers control and tracking means offered by PeSIT, are undoubtedly much more advanced and rich than those offered by other file transfer protocols.

Although the PeSIT protocol is asymmetric, its use as two distinct applications; one, a client, and the second, a PeSIT server, did not emerge. File transfer monitors, which often evolve into multiprotocol solutions, create a network topology where partners take on dual roles as both requesters and responders, across multiple connections that may occur simultaneously or sequentially. The use of the concept of virtual files, closely linked to business applications using these files and the partners, allowed for distinguishable classes of manageable transfer flows. During the execution of IT processing chains, transfer requests were placed in the monitor's transfer queues to be selected according to predefined scheduling rules based on their priorities, their virtual files, or partners choice. They were executed asynchronously with respect to production chains. Interoperability with systems was ensured by executing pre and post-transfer procedures attached to virtual files. An online processing at the reception or reading of each record was also allowed in some products.

Apart from file transfer monitors, specialized platform solutions for financial services, corporate accounting management, and others, also implemented PeSIT OFF-SIT, primarily due to regulation and then for the convenience of its use.

The arrival of distributed processing technologies has led to the emergence of multi-protocol MFT (Managed File Transfer) solutions, offering platforms to orchestrate, secure, and control file transfers within automated business process frameworks. In this context, file transfers can be executed synchronously with processing chains, though asynchronous execution is also possible. Increasingly, MFT solutions support the PeSIT OFF-SIT protocol.

The intrinsic qualities of the PeSIT protocol are such that, more than two decades after the publication of its latest version (still accessible at www.pesit.org^[1]), there are still thousands of users who do not consider using other protocols.

The MFT market is constantly growing, as is the number of file transfers within large enterprises. With the evolution of technologies, new needs have emerged, such as UNICODE file transfers or the transfers of serializable objects, even those stored in the Cloud, which were not considered at the time the protocol was defined. Some publishers leverage the customization capabilities of PeSIT services by transmitting additional information via user parameters. However, this wise process is only applicable to transfers between products from the same publisher.

Today, an increasing number of MFT solutions support the PeSIT protocol, facilitated by the emergence of PeSIT connectors in Java, such as the one from OTONET, already implemented in several products of this type.

Recently developed products can more easily incorporate new needs, but their direct consideration by the protocol would be the best solution.