An API (or Application Programming Interface) allows applications to talk to each other. An API specifies the requests, responses, message conventions, and data formats that should be used.
What is an API? What does it do?
APIs simplify implementation by abstracting the underlying functionality and only exposing data and actions that the developer needs. The details of how the functionality
is accomplished is left to the service; the calling application does not need to understand what actions are required "behind the curtain" to accomplish the outcome.
For example, an online shopping application can use a payment API to authorize payment by specifying transaction details (e.g., amount). The online shopping application
is not exposed to sensitive cardholder data, as it is encapsulated with the payment service.
Why are APIs useful?
REST (Representational State Transfer) is an architectural approach that relies on stateless operations using existing protocols.
A RESTful API is an API that uses HTTP requests to GET, POST and DELETE data.
Interested in learning more? There are many free tutorials on REST APIs. Here are a few to get you started:
What is a REST or RESTful API?
The OpenAPI Specification (OAS), originally known as the Swagger Specification, is a specification for machine-readable interface files for describing, producing, consuming, and
visualizing RESTful web services. Swagger UI is an open source project to visually render documentation defined with the OAS specification. Conexxus and IFSF API standards use OAS V3.
What is OAS? What is Swagger? What is Swagger UI?
YAML (YAML Ain't Markup Language) is a human readable superset of JSON. While JSON is already human readable, YAML makes editing easier and allows insertion of comments, which JSON does not.
YAML files can be read with appropriate software programs or they can be rendered into a human friendly format using Swagger UI (or comparable tools, like ReDoc).
Conexxus members may view files from Conexxus and Open Retailing in a Swagger renderer on the Conexxus website wherever you see a filename followed by "swagger" (under My Stuff/My Documents or Standards).
IFSF and Conexxus members may also view Open Retailing files at docs.openretailing.org
For additonal information on YAML, visit:
What is YAML? How do I read it?
Conexxus and IFSF have published four guidance documents on the following topics (click on each link to download the latest published version):
These are living/working documents. If you are a member of Conexxus or IFSF, you will potenitally find proposed versions in the Open Retailing gitlab under the Work in Progress group/API Design Guidelines project.
If you'd like to give feedback on the guidelines and are a member, please raise an issue in the work in progress project.
If you are not a member and would like to comment/provide feedback on the published versions, please send an email to support@openretailing.org
Are there API Design Guidelines? Where do I find them?
Security has been considered in the design of all Conexxus/IFSF APIs. There are three mechanisms that have been adopted:
OAuth2: This is an industry-standard authorization framework. It keeps user information within the authorization system;
there is no need to store credentials at the client. More suitable for Cloud-based systems, but harder to implement at a
single store location or site only systems. The actual authorization is performed using a random access token. OAuth2 defines
multiple flows supporting different scenarios. For more technical information, refer to the OAuth2 web page: https://oauth.net/2/
Basic Authentication: Built into HTTP protocol, it relies on a username and password credentials. It is
considered basic because it was the first authorization and authentication system implemented natively by the browser,
and it provides a basic username and password authentication. There is no mechanism to support access control for API
level. In Basic Authentication, the user name and password is base 64 encoded and added to the HTTP header "Authorization."
For more information, please refer to the Mozilla page: https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication.
API Key: The API Key is a secret shared across two end points. The API Key is transmitted using a HTTP header “X-API-Key”
and it can be used in conjunction with OAuth2 or Basic Authentication. The open retailing APIs are built so the API Key can be
rotated.
I heard there was no security in the Conexxus/IFSF APIs? Is that true?
An endpoint denotes a URI (or URL) at which you can access an API.
What's an endpoint?
A mock server is a test sever that is useful for injecting errors and other situations so the client (calling) code can be tested effectively.
What's a mock server?
Server Sent Events are part of the "HTML5" specification. They allow a client application to subscribe to an event stream from a
web (HTTP) server. For our RESTful Web Service APIs, services are hosted on HTTP servers, and SSE is available. A key benefit of SSE
is that although the events "stream" from the service host to the client application, it is the client application that instigates the
connection, thus eliminating many PCI issues. SSE is a great technology for notifying listeners when a database changes (e.g., a change
in a price book, a new transaction added from a real-time feed).
The Open Retailing API Design Guidelines document
Design Rules for APIs OAS3.0
gives standard ways to implement an event stream request in an Open Retailing or Conexxus
approved Standard API. It also explains how to define JSON objects to travel over the event (SSE) connection and how to implement
those definitions in Open API Specification 3.0 (OAS 3.0) files.
What are Server Sent Events (SSE)?
Web sockets are a way to open two-way communication between a client application and a service host. The data to be sent over the connection
is not defined – it’s "free format." Common uses for a web socket include streaming videos or sending very large files. For instance, an API
might allow a client to request a Web Socket to see a live camera feed, or see a prerecorded video. Another use could be for sending or
receiving a complete "price book" or other large artifact that might be clumsier to send over HTTP.
Note: some developers think of web sockets and server sent event (SSE) channels as interchangeable. That’s not really true – the SSE
definition provides an interoperable "disciplined" approach to defining events, and web sockets provide the freedom necessary to solve
problems that go beyond providing an event stream.
The Open Retailing API Design Guidelines document
Design Rules for APIs OAS3.0
gives standard standard ways to implement a web socket request in an Open Retailing or Conexxus approved Standard API.
Donated API Work can be found in Gitlab under the Work In Progress Group. Every project defined in Gitlab containing the labels (submitted, under-review or unassigned)
is donated work. The labels represent:
submitted - The work has been submitted for review.
under-review -A Conexxus/IFSF staff is reviewing the API.
unassigned - The API has been reviewed, it conforms to the Conexxus and IFSF Design Guidelines, and is now ready to be reviewed by both
organizations for acceptance as a New Work Item. Conexxus and IFSF will follow their respective procedures for New Work Items.
Where do I find work that has been donated to Conexxus and IFSF for consideration as a future standard?
Git/Gitlab
Git is a distributed version control system that uses "repositories" to track all changes made within a project. In other words,
a repository is the file that maintains the history of the project.
Instances of git run on a server application such as github, gitlab, bitbucket. These applications are all very similar in that they
interact with the git repositories in the same fashion, but some have
added features on top of the git features such as issue tracking and code review. Conexxus and IFSF use gitlab as the server application
and the instances live here:
gitlab.conexxus.org and gitlab.openretailing.org
Note: You cannot login to directly to gitlab.conexxus.org at this time.
Gitlab is a hosted environment that provides additional features for using Git collaboratively (i.e., as a group), such as notifications,
browsing between projects, and issue tracking. Logging in directly is the way to access these features. However, development and commits
are not recommended using this interface. Instead, use either of the following access methods:
Command line: This option may be the best choice for users with git experience or programming experience with command line interfaces.
Client with GUI: This option is the best choice for users that are new to git or would prefer to do tasks via a
graphical user interace. There are numerous git clients commercially available (GitHub Desktop, TortoiseGit, GitKraken, SourceTree, etc.) If you do not already have a favored client, Conexxus recommends and offers support for SourceTree (which is free). Other FAQs discuss SourceTree in more detail.
What is Git? What's a git repository? What is Gitlab? How do I access/interace to it?
GitLap supports the SSH (Secure Shell) protocol for communications between Git and its servers. An SSH key pair (one private and one public)
is used to authenticate users. Your public SSH key must be set in the gitlab server, which will allow access to anyone who has the corresponding
private key. You should never give your private key to anyone.
You will need to install Git to generate an SSH key.
Video: How to install Git and Generate SSH Key - Windows
NOTE: After installing Git for Windows and generating your public key, you must email your public key to
support@conexxus.org for the Conexxus gitlab instance. You can setup your own key directly in the Open Retailing instance or email it to support@openretailing.org and we will set it up for you.
You will receive a confirmation email after it has been set up and you can then install and configure a client (command line or gui) of your choice.
What's an SSH Key? Do I need one? How do I create one? What do I do with it?
Creating Multiple SSH Keys: If you are already using SSH keys to connect to another server, you can create a new key pair to connect to Conexxus, and then tell SSH to use those keys for Conexxus.
To create a new key, use:
ssh-keygen -b 4096 -f ~/.ssh/conexxus
You will need to send your new public key (found at ~/.ssh/conexxus.pub) to support@conexxus.org (for the conexxus instance of gitlab) and support@openretailing.org (for the openretailing instance of gitlab) to be added to our system.
Then add the following to your ~/.ssh/config file to assign the keys:
Connection Problems: If you receive an error like "Connection Reset By Peer" this typically
indicates that the SSH connection to the gitlab servers are being blocked. Contact your IT department and ask them to
"Unblock your SSH Connection on Port 22 to gitlab.conexxus.org" and "Unblock your SSH Connection on Port 22 to gitlab.openretailing.org"
and this should solve the problem. Alternatively, you could temporarily connect to a hotspot or other network to get around the firewall blocking (assuming this is permitted by your IT department).
What if I already have an SSH Key for my work (or other) server? What if I get a Connection Reset by Peer error?
The Conexxus and OpenRetailing GitLab instances are privately hosted. Therefore, your public gitlab accounts are not authenticated on these servers.
Instead an account will be created based on your Conexxus or IFSF logins. Each gitlab instance will have a unique password which is different than your website login.
You may choose to use the same password across accounts.
Many accounts are generated automatically. But, if you find you need access to the gitlab instances:
Please send requests for the Conexxus gitlab instance to support@conexxus.org
Please send requests for the OpenRetailing gitlab instance to support@openretailing.org
How do I get access to the gitlab servers? I already have a gitlab account, why do I need a new one for this work?
If you want to use the command line interface, please refer to the Git Manual. Specifically chapter 2 provides a good introduction to the basic commands you will need to use.
Video: How to install SourceTree for Windows (using Git for Windows)
How do I install SourceTree? How do I load my SSH key into SourceTree?
Everything in gitlab is organized into groups and projects. Groups are created based on work stream and user permissions. Projects belong to groups.
Video: How to Find Project in Gitlab
Where do I find project repositories in gitlab?
Within each project repository, there is a readme file. This is a great place to look to see what's how the project is organized and where files can be found.
Video: How to Locate the Readme File for a Project
How do I navigate a repository? Where do I find things I'm looking for?
A clone is a copy of the remote repository that lives on your local machine.
Cloning a repository allows you to work with files on your local machine. This prevents your work from affecting others using the same repository until your changes are complete and pushed back to the remote repository.
Video: How to find the SSH URL and clone a repo
What is a clone? How do I clone a repository?
Commit, Push: Git maintains a history of all changes that have occured within a repository. A commit is the basic unit of work that enables the maintenance
of that history. When local development work is completed, you commit that work to your local copy of the repository. A push involves
publishing one or more of your commits to the remote repostiory. Your commits remain on your local machine and
other people using the repository will not see your changes until a push of those commits is complete. All other git artifacts
depend in some way on a commit having occurred.
Branch: Each repository starts with a "master" branch. Additonal branches can be created that are separate work areas that are "forked" off of a
previous commit. A branch is a moveable pointer to a snapshot of committed files (i.e., the pointer moves as more files are committed to the
branch.) The branch remains separate until a merge back into another branch (usually, but not always, master) is completed.
Branches are used to isolate development work, perform bug fixes, and create parallel work streams.
Tag: A tag is a pointer to a snapshot of committed files. Unlike branches, tags do not move as additional commits are made into the repository.
Tags are used to mark status of a certain commit (e.g., "ready-for-review") or most commonly, you’ll find them on “master” branches as
indications of a version release (e.g., V1.0, V2.0).
Version tags can be used in the file dependencies.txt to reference specific versions of referenced repositories required (e.g., the data dictionary.)
Merge: Once a branch has been created and all work completed, that work can be merged to the original branch or even some other branch.
If you worked in the recommended manner (i.e., making changes first on your local machine), you must push your work to the remote repositiory on
the GitLab Host before merging. Most of the time, the changes on the branch being merged are straightforward with regard to the target
branch. If changes occur on the target branch that affect the same files changed on the branch to be merged, conflicts may occur.
These must be resolved during the merge. In any case, the end result is all the changes are merged on a single branch.
Merging is normally, but not always, done on the GitLab Host. In GitLab,
merge operations are most often achieved through a “merge request” which provides for notifications and recording of the reasons for the
merge.
Video showing how to locate branches and tags in Sourcetree:
What is a commit? A push? A branch? A tag? A merge?
In brief, when an problem is found or a new feature is desired, here are the steps that should occur:
What is the recommended git work flow?
Issues are used to request changes. Issues should only be raised on work in progress projects.
Video: Create an issue
Video: Comment on an open issue
What are issues? How do I create an issue? How do I comment on an open issue?
To associate a branch with an issue, start the name of the branch with the issue number, followed by a non-numeric character (e.g., '-').
Video: Create a branch to do development work. Each branch should have an associated issue.
Video: When you are done with development on a branch, raise a merge request which triggers peer review and if no problems are found, it is merged back to master (by staff admins).
How do I create a branch to do dev work against an issue? How do I create a merge request?
In order to participate in group on-line voting on changes, you MUST subscribe to notifications. This video explains how to do that.
How do I set notifications in gitlab?
API Data Dictionary
The API Data Dictionary contains uniquely defined objects and elements used in Open Retailing APIs. It is intended that like data is reused across APIs.
The dictionary is an ongoing work in progress that can be accessed by members and can be found at: https://gitlab.openretailing.org/work-in-progress/api-data-dictionary
Members are requested to inform us of missing data elements and objects, as well as requests for corrections, additions, improvements, and clarifications of the existing elements and objects.
All proposed changes will be reviewed by work groups within Conexxus and IFSF, and potentially be accepted into the Data Dictionary.
Changes and additions are reviewed to assure consistency throughout the dictionary and to avoid duplications of the same name being used for the identical element or object.
The publicly released V1.0 can be found here. Future versions may be released to the public.
What is the API Data Dictionary? Where do I find it?
Traits are objects that don't quite belong in the data dictionary, but they are still useful and can be re-used.
For instance, most reports that contain multiple records will require a "start" and a "limit". Look at those as framework objects that
will avoid redefining the same data.
Traits are located in a separate folder within the api-data-dictionary repo called "traits".
What is a trait?
The utilities directory in the API Data Dictionary repository provides various definitions that are not specific to a single API.
For instance, error codes, how to format a "heartbeat" message, and standards for response value are examples of utilities.
What are utilities?
API Groups that reference the data dictionary are written at various times, and the dictionary versions may change.
When reviewing an API Group is important to know what version of the API Data Dictionary should be used.
You can find this value by looking in the api/dependencies.txt directory in the project. The end of the line should have the version.
What is a dependency on the API Data Dictionary?
Tools
Openretailing.org currently contains a comprehensive set of Simulators and Certification tools. For more information, email support@openretailing.org.
Below is a list of a few Simulators and Certification Tools currently available.
Dispenser
Carwash
Mobile 1.0 (Remote Fueling Approval)
Digital Offers
PARA - POS Activity Report API
Site Asset Data
FDC
What simulators are available? Where do I find them?
Please, send an email to: support@openretailing.org
How do I run a simulator?
Conexxus/IFSF has no specific requirements. Some popular free tools are:
What editor and/or integrated develop environment does Conexxus/IFSF require?
Postman is a very popular tool for testing and documenting APIs. If you've developed an API, Postman can create a
small test server (mock server) for you. If can also help you create scripts to test it.
See https://www.postman.com/
Postman is the primary API Client tool used by the API Factory. By using Postman,
users will be able to preview and exercise APIs before they initiate development.
What is Postman? When would I use it?
Node.js is a JavaScript interpreter/VM. Node.js is required to run some of the tools and simulators.
To install it, navigate to https://nodejs.org/en/, download the installer for your platform and run the executable.
Video: Installing Node.js
What is Node.js? Why do I need it? How do I install it?
NPM stands for "Nodejs Package Management" and it is the tool used to install nodejs modules and applications. For instance, to
install the swagger-cli, just type the command npm -g install @apidevtools/swagger-cli. In order to run the OpenReatiling.org
API validator, you must first install nodejs and npm. Once those tools are installed, run the command "npm install" at the
validator’s root directory. The command "npm install" will read the package.json file and retrieve and install all the dependencies.
What is the Nodejs Package Manager (NPM)? Why do I need it?
Express is a web framework for Node.js. With Express, Node.js applications can be self-contained and don't have to rely on
an external web server. In additional, Express provides additional middleware functionality such as the ability to handle
errors, parse cookies or perform authentication. For additonal information, see https://expressjs.com/
What is Express? Why do I need it?
Docker is a containerization tool. One of the advantages of using the "Cloud" is that it can perform Horizontal Scaling, i.e.,
if the traffic to an application increases, more instances of that application can be "spawned" to support the extra load. If the usage goes
down, instances of the application can be terminated. Before Docker, the tools used to perform horizontal scaling were Virtual Machines (VMs).
VMWare, VirtualBox, Linux KVM and other widely deployed virtualization systems work by virtualizing the entire computer. Virtualization of
the entire computer is computationally expensive, as every single component of a physical computer device must be virtualized (CPU, HD, Memory,
peripherals). With Docker, only the OS is virtualized and for that reason Docker is considered lightweight. A given computer device can handle
many more instances of a Docker container vs Virtual Machines.
What is Docker?
In Conexxus and Open Retailing GitLab environments, we are using the "Continuous Integration/Continuous Deployment" (CI/CD) tools
provided by GitLab, called "GitLab CI/CD Pipeline Runner". In general, CI/CD tools automate the process of building and deploying
an application: download dependencies, build applications, run tests and then deploy required code to a production server. For
instance, we can use CI/CD to build the documentation every time the OpenAPI definition file changes.
What is CI/CD?
The API specification validator is an automated tool to ensure that an API conforms to the API Design Guidelines.
Video: API Specification Validator
The OpenRetailing API Specification Validator is also available in the public "npm" repository. When nodejs and npm are installed,
the API Specification Validator tool can be installed using the command "npm install -g @openretailing/spec-validator".
For more information, refer to the site: Spec-Validator
Note: the spec validator returns a lot of useful information, but the version of the validator may lag some with regard to the latest API Design Guidelines.
What is the API Specification Validator? Where is it? How do I use it?
Conexxus is a non-profit, member-driven technology organization dedicated to the development and implementation of standards, technologies innovation and advocacy for the convenience store and petroleum market. Conexxus membership collaborates on key present and future industry challenges and innovations. Our work efforts improve profitability by reducing the cost of IT ownership and improving the competitiveness of our members.
About IFSF
IFSF is a community dedicated to enabling technology standards for the benefit of retailers of automotive fuel and energy. IFSF's approach is to collaborate with other standards organisations and relevant institutions to achieve common standards for interoperability of forecourt and convenience devices and services. IFSF depends on support from retailers, suppliers and other industry stakeholders who contribute to, and benefit from, shared standards. To date IFSF has set many standards which have been adopted on thousands of forecourts in an ever increasing number of countries world wide.
