Trading of commodities is an extremely complicated and equally well
researched area. One reason why it is so popular is that Procurement
teams are always under pressure to bring down procurement costs
and success in solving the trading puzzle could translate directly into
earned or saved dollars. Very recently, my team was thrown the challenge
for coming up with solution for a specific commodity price prediction.
The problem made us to explore innovative solutions in areas of machine
learning, agent based modelling, cognitive cognition and last but not
the least economics.
Demand and Supply of a Commodity : You would find tonnes of complicated definitions for demand, but in simple words Demand is the quantity people want to buy and supply is the quantity produced by manufacturers which they want to sell in market. Price is where these two graphs intersect each other. How easy is to infer that any increase in supply would cause prices to drop and any increase in demand would cause them to price. This is what the classic economics teaches us.
Now we would not need Machine Learning models and Cognitive technologies if situation on ground was that simple. The actual traded amount may be entirely different from demand and supply intersection point. In a supply strong economy (supply surplus), which is often characterized by falling prices, suppliers may decide to cut-down on the quantity supplied, exactly what OPEC nation does when price of Crude nosedive, they put artificial restrictions on production and thus supply in market. While this may lead to prices increasing temporarily, in long run it may lead to a cut-back in the quantity demanded. Commodities whose demand side is stronger than supply side can lead to very interesting economic situations. So gist of all of this, The "short side" of the market dominates, with limited quantity demanded constraining supply in the demand constrained scenario and limited quantity supplied constraining demand in the supply constrained scenario.
Generally, the markets short side is the demand side. How many times have you witnessed a scenario where there is shortage of shirts when you want to buy them, or how about that dream car. Quantity supplied is nearly always more than quantity demanded, reason why MBA folks in sales make a tonne more than others (or do they ?). Reality is a salesman would only be happy to make available as many shirts/laptops/consoles as you would want to buy, think you are getting hang of markets in general now. In such economy if you increase quantity supplied, without increasing quantity demanded, price would have zero or minimal impact. But if you increase quantity demanded, even if you don't increase quantity supplied, the quantity bought and sold will increase, pushing up the prices. For some rare commodities however, supply is the constraining side. Specially in Chemical industry, there are lot of examples particularly for those which are downstream of Crude Oil. These markets behave in complete contrast to normal markets. At Industry level, instead of spot procurement, we tend to see long term contracts between buyers and suppliers for the quantity to be traded, however purchase price component may not be fixed and that may be actually decided on delivery date. In these markets seller and buyer relations matter a lot, since seller can easily sell of his produce at spot prices to a different buyer making more money than he would have done by selling as per contract. Buyers also sometimes need to buy from spot market to achieve their procurement quotas and need a mechanism to calculate a fair spot price for such commodities.
In today's connected world, where no industry is an island, there could be altogether different reasons which can upset local demand and supply equilibrium in such economic situations, like sudden spike in demand from foreign markets, increase of decrease in downstream derivatives of commodity, environmental issues causing change in commodity ecosystem, change in political scenario may cause prices to become extremely volatile etc.
What one can infer from above statement is that a true commodity prediction model would need inputs from various diverse and often disconnected data sources and domains. Traditionally this job is left to procurement specialists, who in turn read reports from experts belonging to one of the above domains. It is then left to experience of procurement specialist to make sense of all the diverse signals, bring them together to predict price variations in a commodity. In the software industry, many models like distributed ML, ensemble ML etc. have been used to address this diversity, with far from satisfactory results.
Our team as such decided to build a new Collaborative Cognition (CC) platform, to help converge different price predictions from diverse set of machine learning agents(representing different knowledge worlds) into a single predicted price range. Additionally it helps these autonomous agents become aware of some of features being used by other participating agents, making their individual predictions even better. As CC keeps checking on accuracy of predictions from these individual agents, by comparing real commodity prices with their predictions, it can over the time start assigning different importance weights to these agents, bringing the predicted range very close to real commodity prices. Combined with innovative user interfaces, not only allows a procurement specialist to visualize predictions, but also why and how those predictions were made. He can switch to view of individual agents view and see what is happening in markets from a specific perspective eg political or regulatory or pure econometric.
More on Collaborative Cognition and use cases IBM IRL team is solving using it, in coming blogs .....
*Note: Thoughts, opinions and expressions made in this article belong to the author alone. They are not reflective of any organization or company to which he may be associated or employed with, neither directly nor indirectly.
Demand and Supply of a Commodity : You would find tonnes of complicated definitions for demand, but in simple words Demand is the quantity people want to buy and supply is the quantity produced by manufacturers which they want to sell in market. Price is where these two graphs intersect each other. How easy is to infer that any increase in supply would cause prices to drop and any increase in demand would cause them to price. This is what the classic economics teaches us.
Now we would not need Machine Learning models and Cognitive technologies if situation on ground was that simple. The actual traded amount may be entirely different from demand and supply intersection point. In a supply strong economy (supply surplus), which is often characterized by falling prices, suppliers may decide to cut-down on the quantity supplied, exactly what OPEC nation does when price of Crude nosedive, they put artificial restrictions on production and thus supply in market. While this may lead to prices increasing temporarily, in long run it may lead to a cut-back in the quantity demanded. Commodities whose demand side is stronger than supply side can lead to very interesting economic situations. So gist of all of this, The "short side" of the market dominates, with limited quantity demanded constraining supply in the demand constrained scenario and limited quantity supplied constraining demand in the supply constrained scenario.
Generally, the markets short side is the demand side. How many times have you witnessed a scenario where there is shortage of shirts when you want to buy them, or how about that dream car. Quantity supplied is nearly always more than quantity demanded, reason why MBA folks in sales make a tonne more than others (or do they ?). Reality is a salesman would only be happy to make available as many shirts/laptops/consoles as you would want to buy, think you are getting hang of markets in general now. In such economy if you increase quantity supplied, without increasing quantity demanded, price would have zero or minimal impact. But if you increase quantity demanded, even if you don't increase quantity supplied, the quantity bought and sold will increase, pushing up the prices. For some rare commodities however, supply is the constraining side. Specially in Chemical industry, there are lot of examples particularly for those which are downstream of Crude Oil. These markets behave in complete contrast to normal markets. At Industry level, instead of spot procurement, we tend to see long term contracts between buyers and suppliers for the quantity to be traded, however purchase price component may not be fixed and that may be actually decided on delivery date. In these markets seller and buyer relations matter a lot, since seller can easily sell of his produce at spot prices to a different buyer making more money than he would have done by selling as per contract. Buyers also sometimes need to buy from spot market to achieve their procurement quotas and need a mechanism to calculate a fair spot price for such commodities.
In today's connected world, where no industry is an island, there could be altogether different reasons which can upset local demand and supply equilibrium in such economic situations, like sudden spike in demand from foreign markets, increase of decrease in downstream derivatives of commodity, environmental issues causing change in commodity ecosystem, change in political scenario may cause prices to become extremely volatile etc.
What one can infer from above statement is that a true commodity prediction model would need inputs from various diverse and often disconnected data sources and domains. Traditionally this job is left to procurement specialists, who in turn read reports from experts belonging to one of the above domains. It is then left to experience of procurement specialist to make sense of all the diverse signals, bring them together to predict price variations in a commodity. In the software industry, many models like distributed ML, ensemble ML etc. have been used to address this diversity, with far from satisfactory results.
Our team as such decided to build a new Collaborative Cognition (CC) platform, to help converge different price predictions from diverse set of machine learning agents(representing different knowledge worlds) into a single predicted price range. Additionally it helps these autonomous agents become aware of some of features being used by other participating agents, making their individual predictions even better. As CC keeps checking on accuracy of predictions from these individual agents, by comparing real commodity prices with their predictions, it can over the time start assigning different importance weights to these agents, bringing the predicted range very close to real commodity prices. Combined with innovative user interfaces, not only allows a procurement specialist to visualize predictions, but also why and how those predictions were made. He can switch to view of individual agents view and see what is happening in markets from a specific perspective eg political or regulatory or pure econometric.
More on Collaborative Cognition and use cases IBM IRL team is solving using it, in coming blogs .....
*Note: Thoughts, opinions and expressions made in this article belong to the author alone. They are not reflective of any organization or company to which he may be associated or employed with, neither directly nor indirectly.
