ONDC: Democratizing Digital Commerce

            ONDC, short for Open Network for Digital Commerce, aims to revolutionize digital commerce by democratizing access. Unlike traditional e-commerce platforms like Amazon or Flipkart, ONDC operates as an open network, allowing both buyers and sellers to interact regardless of the platform they use. This inclusivity is expected to benefit various industries such as retail, food, and mobility. 

        By standardizing operations, providing logistics support, and digitizing the entire supply chain, ONDC represents a significant shift away from the platform-centric model. Instead, it fosters a facilitator-driven, interoperable, and decentralized network. This move challenges the dominance of major platform players like Flipkart and Amazon, promising to level the playing field for all participants in the digital commerce landscape.

On the ONDC Network, there are three types of Network Participants:

1.Buyer Apps (Buyer Nodes):

·      Responsible for buyer-side operations such as acquiring buyers, facilitating search and discovery, and enabling buyers to place orders on the Network.

2.Seller Apps:

Manage seller-side operations and come in two varieties:

a. Marketplace Seller Nodes (MSNs):

·   Function as aggregators, allowing sellers who are not direct network participants to conduct transactions on the Network.

·   MSNs do not maintain inventory but serve as marketplaces for sellers.

b. Inventory Seller Nodes (ISNs):

·   Are sellers themselves who actively participate in the network.

·   These sellers maintain their inventory and transact directly on the Network.

3.Gateways:

·    Serve as nodes for broadcasting search queries and collecting results.

·    Play a vital role in search and discovery processes, facilitating efficient interaction between buyers and sellers.

ONDC’s Architecture:

In any individual sale facilitated through ONDC, the Buyer and Seller are bound by the terms outlined in the transaction. The transfer of goods' title occurs from the Seller to the Buyer via the invoice.

Both the Buyer and Seller have pre-existing legal relationships with their respective apps, governed by the terms and conditions set by those apps. However, there's no direct legal relationship between the Buyer and Seller apps.

Instead, the Buyer App and Seller App each have a legal relationship solely with ONDC, established through the Network Participant Agreement (NP Agreement). While this agreement establishes the legal framework, the operational aspects of transacting on the ONDC Network are detailed in the ONDC Network Policy.

Before a transaction, the Buyer App and Seller App don't have a pre-existing legal relationship. A legal relationship is dynamically formed between them through the Transaction-level Contract, facilitated by the ONDC Protocol. This contract is executed digitally between the Buyer App and Seller App, with ONDC not being a party to the transaction itself.

Retail API contract includes the following APIs:
a. Pre-order APIs
·         /search - buyer app specifies the search intent; 
·         /on_search - seller app responds with the catalog based on the search intent;
·         /select - buyer app specifies the items & quantity selected by a buyer; 
·         /on_select - seller app responds with the serviceability info, quote & O2D TAT for selected items; 
·         /init & /on_init - buyer & seller app specify & agree to the terms & conditions prior to placing the order; 
·         /confirm - buyer app places the order on behalf of the buyer; 
·         /on_confirm - seller app responds to the order placed either through auto-acceptance or deferred acceptance or rejection of the order;

b. Post-order APIs 
·         /status - buyer app requests for the current status of the order; 
·         /on_status - seller app provides the current status of the order; 
·         /cancel - buyer app places cancellation request for the order; 
·         /on_cancel - seller app responds to the buyer app cancellation request or cancels the order directly; 
·         /update - buyer app initiates a part return or cancel request (for specific items) on behalf of the buyer; 
·         /on_update - seller app responds to the buyer app part return / cancel request or initiates a part cancel request (for specific items) directly; 
·         /support - buyer app requests seller contact details; 
·         /on_support - seller app responds with seller contact details; 
·         /track - buyer app requests for live tracking of order; 
·         /on_track - seller app responds with URL for live tracking of order; 
·         /rating - buyer app provides rating on behalf of buyer;

• /on_rating - seller app responds to rating request from buyer app and provides additional info, such as feedback form, as required;

Stages in the life-cycle of ONDC transaction:

The following breakdown outlines the various stages involved in a transaction within the ONDC Network, including both standard processes and exception handling.

1. Search and Discovery: The buyer searches for a product and receives a list of sellers offering that product.

·         Buyer initiates a search (product, service, or store) on the buyer app interface.

·         Buyer app interface relays the query to the gateway service provider.

·         Gateway service provider checks within the ONDC registry to identify seller applications who sell the product/service matching the search query. 

·         Gateway service provider multi-casts the search to all relevant seller applications simultaneously.

·         Seller applications check with their registered sellers on availability. 

·         Seller applications communicate availability information to the gateway. 

·         Gateway service provider relays the information back to the relevant buyer application. 

·         Buyer sees responses to his/her search query on the buyer application interface

2. Placing an Order: The buyer selects a product, adds it to their cart, and confirms the order.

Buyer selects the product/service and wants to view product details, price, offers, delivery options, and terms of sale.

·         Buyer application contacts seller application to get the relevant information about the product.

·         Seller application gathers necessary information from the seller.

·         Seller provides product details, quotation, and terms of sale.

·         Seller application relays the above information to the buyer application.

·         Buyer application shows the details to the buyer.

·         Buyer confirms the product, makes the payment, and confirms the order.

·         Transaction-level contract is created between the buyer application and the seller application.

·         Seller application confirms the order.

3. Fulfillment: This stage involves delivering the product to the buyer, including finalizing arrangements for delivery. Unlike the platform-based model where fulfillment typically only includes delivery execution, in ONDC, fulfillment arrangements overlap with the order placement process.

    

•  Buyer initiates a search (product, service, or store) on buyer application interface. 
•  Buyer selects product and confirms delivery terms on product page.
•  Buyer app/seller app with a pre-existing agreement with a logistics buyer relays the above information.
• Logistics buyer searches for logistics service providers (registered on seller app, or MSNs delivery agent, or logistics sellers on network).
• Logistic buyer receives quotation from logistics service provider.
• Logistics buyer confirms the quotation and delivery time to buyer app.
• Buyer app relays this information to the buyer.
• Buyer confirms order to buyer app, shares address, delivery instructions, and makes the payment. Order is confirmed and the logistics service provider relays the turnaround time, details of the delivery agent, tracking link, etc., to the buyer’s order.
• Logistics buyer and logistics service provider sign a transaction-level contract for services.
• Logistics service provider picks up the order from the seller (either an ISNs or MSNs) . 
• Order is delivered to the buyer.

4. Payment and Settlement: The buyer completes payment for their purchase, and entities involved in the transaction receive their dues. While payment usually precedes order confirmation, and settlement occurs after payment, this breakdown simplifies the explanation.

·         Buyer confirms the order and makes the payment.

·         Logistics NP confirms the order and collects payment from the buyer.

·         Reconciliation service provider (RSP) receives settlement terms and amounts from the transaction-  level contract signed between the NPs.

·         RSP prepares settlement advice and sends to NP collector’s settlement agency.

·         Settlement agency initiates settlement through recipient’s bank.

·         Recipient’s bank receives the payment.

·         Recipient NP receives payment from his bank.

5. Returns/Refunds/Cancellations: Although not a formal stage in the transaction lifecycle, these events occur frequently and require a systematic model, policy, and process for efficient handling.

• Sellers will initiate returns only if the goods are found defective or damaged.
• Seller can initiate a refund instead of a return.
• Cancellations will only be possible if the seller has declared it in the transaction-level contract.
• Return logistics will be initiated by the seller or seller app as per the terms agreed in the return policy

 6. Issue and Grievance Management: Not a stage in the transaction lifecycle but rather an exception-handling process. It can arise at any stage of the transaction, most commonly after an order has been confirmed and paid for.

Example of ONDC Transaction:

PostgreSQL Backup Health Check Best Practices

There are several backup strategies available for PostgreSQL, and the right one for your environment will depend on your backup and recovery requirements. Some popular strategies include:
 
1. Physical Backup:
·        A physical backup is a binary copy of the database cluster's files at a specific point in time.
·         It includes all database objects, system catalogs, configuration files, and transaction logs.
·         The backup is essentially a snapshot of the entire database storage.
·         Physical backups are usually faster to create and restore compared to logical backups.
 
2. Logical Backup:
·         A logical backup contains SQL statements that can recreate the database's structure and data.
·         This type of backup is created using tools like pg_dump, which generates a script with SQL                    commands to recreate the database schema and insert the data.
·         Logical backups are more human-readable and can be more flexible in terms of selective data                  restoration.
·         However, they are generally slower to create and restore compared to physical backups.

  

How to verify health of backup image in PostgreSQL

Verifying the health of a backup image in PostgreSQL is crucial to ensure that you can rely on it for recovery in case of data loss or system failure. Here are steps you can take to verify the health of a backup image:
 
1.      Use pg_verifybackup:
Starting from PostgreSQL 13, there is a tool called pg_verifybackup that you can use to verify the integrity of a backup. This tool can check whether the backup is consistent and whether it contains all the required data.
Example:
                pg_verifybackup    /path/to/backup
                The tool will analyze the backup and report any issues it finds.
 
2.      Check Log Files:
Review the PostgreSQL log files generated during the backup process. Look for any error messages or warnings that might indicate a problem with the backup. The log files are usually located in the PostgreSQL data directory, and the main log file is typically named postgresql.log.
 
3.      Restore and Test:
The most reliable way to verify the health of a backup is to restore it to a test environment and perform some basic tests. This will ensure that the backup is not only consistent but also functional for recovery.
·         Restore the backup to a different location or a separate test server.
·         Start the PostgreSQL server using the restored data.
·         Perform some basic queries and checks to ensure the data integrity.
 
4.      Checksums:
Some backup tools and methods support checksums. For example, if you are using pg_basebackup with the --write-recovery-conf option, it generates a backup_label file that contains a checksum. You can compare this checksum with the checksum of the actual backup files to verify integrity.
Example:
                pg_verifybackup --checksums /path/to/backup
 
5.      Use pg_restore:
If you have taken a custom-format dump using pg_dump, you can use pg_restore with the --list option to verify the integrity of the dump file.
Example:
                pg_restore --list /path/to/dumpfile
 
Unlike physical backups (created with tools like pg_basebackup), logical backups created with pg_dump do not have a dedicated health check tool similar to pg_verifybackup. The nature of logical backups makes it more challenging to perform a standalone health check without restoration because the backup is essentially a script containing SQL statements.
  

Creating Physical Backup using pg_basebackup 

&

Verifying Health of the Physical Backup Image using pg_verifybackup

 
Step 1: We can take a physical backup using pg_basebackup command as follows: 

pg_basebackup.exe  -U postgres  -D  D:\verify

 
A backup manifest file with contents will be generated backup_manifest file is nothing but a JSON object, with the following:
      The list of files included in the backup and, for each of them, the relative path from PGDATA and important metadata such as size, time of last modification and checksum        
 
Step 2:  From PostgreSQL 13, there is a tool called pg_verifybackup that can be used to verify the integrity of a backup. This tool can check whether the backup is consistent and whether it contains all the required data.

pg_verifybackup.exe  D:\verify\

 
It will compare the backup_manifest file with the contents generated by pg_basebackup. If all the contents matched properly including checksum then it will generate a message backup successfully verified otherwise invalid checksum for file and its details will be displayed.
 
·         Backup verification proceeds in four stages. First, pg_verifybackup reads the backup_manifest file. If that file does not exist, cannot be read, is malformed, or fails verification against its own internal checksum, pg_verifybackup will terminate with a fatal error.
·         pg_verifybackup ensures the accuracy of data files in a PostgreSQL backup, with exceptions for certain files (postgresql.auto.conf, standby.signal, and recovery.signal, because it is expected that these files may have been created or modified as part of the process of taking the backup )and an indirect check for missing directories based on the presence of expected files.
·         Next, pg_verifybackup will checksum all the files, compare the checksums against the values in the manifest, and emit errors for any files for which the computed checksum does not match the checksum stored in the manifest.
·         Finally, pg_verifybackup will use the manifest to verify that the write-ahead log records which will be needed to recover the backup are present and that they can be read and parsed.
 

Creating Logical Backup using pg_dump 

&

Verifying Health of the Logical Backup Image

 
Step 1: We can take a logical backup using pg_dump command as follows: 

pg_dump -U postgres -d employ -F c -f d:\verify\backup_file.dump

 
-U postgres: Specifies the PostgreSQL user
-d employ: Specifies the name of the database
-F c: Specifies custom format is a compressed binary format output file
-f d:\verify\backup_file.dump: Specifies the output file
 
Step 2:  To verify the integrity of a backup file, typically use tools like pg_restore for custom format backups or other methods appropriate for the backup format (-F option with pg_dump). To verify a custom format backup using pg_restore:

 pg_restore --dbname=your_database --list d:\verify\backup_file.dump

 
This command lists the contents of the backup without actually performing the restore. If there are no errors reported, it indicates that the backup file is in good health.
 
Note: pg_restore -l d:\verify\backup_file.dump   (or)
           pg_restore --dbname=employ --list d:\verify\backup_file.dump  “--can be used”.
 

***Very Very Important Note***

For a more thorough health check of a logical backup, you may want to consider additional steps:
 
Step 1: Perform a Dry Run Restore:  This involves actually restoring the backup to a temporary or test database. This process helps ensure that the backup can be successfully restored and that there are no unforeseen issues.

 pg_restore --dbname=your_test_database backup_file.dump

Step 2: Schema and Data Checks: Generate separate SQL scripts for the schema and data portions of the backup using pg_restore. 

pg_restore --dbname=your_database --schema-only backup_file.dump > schema.sql

pg_restore --dbname=your_database --data-only backup_file.dump > data.sql

Review these scripts to ensure that the schema and data are correctly represented.
 
Step 3: Checksum Verification (if applicable): If checksums are enabled in your PostgreSQL cluster, use pg_checksums to verify checksums in the data directory.

 pg_checksums -D /path/to/your/data_directory

Step 4: Review Logs and Output: Check for any warnings or errors in the output of the pg_restore command and review the logs.
 
Remember that these checks are not exhaustive, and the effectiveness of your health check process may depend on the specific requirements and characteristics of your database. Always make sure to test your backup and restore procedures in a controlled environment to ensure that you can reliably recover your data when needed.