Finding patterns in prices
Capturing action-reaction chains
Action and reaction
There is a famous stock market adage that "stock prices are the realm of response, not a prediction." It may be referring to the importance of risk management by viewing response as risk management, but to me, this maxim seems to be talking about an important concept beyond that.
The response is also broadly included in the realm of prediction. If you buy stocks in response when the market plummets, this is more like a prediction that the stock price will rise rather than the current price in the future. However, this response is not a vague directional bet, but a prediction of a future price movement (reaction) for a specific event (action), and it is also true that many patterns that enable such prediction exist in the market. Qraft Technologies, where I work, is also a company that researches methodologies to capture these kinds of patterns through AI technology.
Although price is a result of various causes, the most interesting and important characteristic of the financial market is that the price movement (action) itself acts as a very important factor leading to the subsequent price movement (reaction). All market participants contribute to the price movement by executing their own decisions, but they are greatly affected by the price movement and test and revise their hypotheses by constantly receiving feedback from the market price. This is a phenomenon common to all markets, not just the stock market, and is the biggest source of alpha. It can be said that George Soros' theory of reflexivity is also broadly the same.
There are a lot of patterns in which subsequent price movements are affected by price movements. This pattern mainly appears in the form of momentum or mean regression, and in the case of strategy C, which appeared in the AI asset management report, it is quite good when short selling stocks that have risen sharply every day at the rising ratio and buying the stocks that have fallen the most at a falling ratio. This is a mean reversion strategy that yields results.
In this article, we are going to look at examples of patterns that can give us a closer look at the meaning of the price structure of the stock market.
Patterns of gap returns and intraday returns.
Most stock markets have openings and closings, and in the case of the New York Stock Exchange, a bell rings every day at opening and closing to commemorate the opening and closing ceremony. The Qraft team also rang the NYSE closing bell when we were invited to NYSE while listing the Qraft AI-driven ETF lineup that is managed by AI without human intervention.
There is a considerable amount of time between closing and next day opening, and information generated during this time term is reflected at once as a price change at the time of opening. This price change is called a Gap. When we say the gap up, the opening price starts when it rises from the previous day's closing price, and when we say the gap down, it starts when the market price falls below the previous day's closing price. So, the daily return (P) is the sum of the gap return (G) and the intraday return (D).
P = G + D,
More precisely, with compound concept, (1+P) = (1+G)*(1+D).
A well-known pattern is that a gap usually has a positive return and an intraday return has a negative return. This pattern appears with high probability in stock markets around the world, especially in small-cap stocks. Although the sign and direction of the gap return and intra-day return is somewhat different for each index, a similar phenomenon is observed in many indices, including the S&P index dating back to the 1920s and the Nikkei index calculated from 1949.
Cumulative return of KOSDAQ index, G and DThe bottom of the graph above is a graph drawn by taking a daily candlestick graph of the KOSDAQ Composite Index from 2010 to the present, decomposing the daily returns into the gap return G (blue line) and the intraday return D (pink line) and accumulating them respectively. The graph of the KOSDAQ index daily return P accumulated (almost the same as the KOSDAQ composite index graph) looks random, but if you look inside, the gap return G has a fairly strong positive tendency, and the intraday return D has a fairly strong negative. It can be seen that there is a tendency In the case of the KOSDAQ index, it continued to rise through the gap and continued to fall during the intraday (stochastically). If you look at the compounded result, it becomes a more dramatic graph. The cause of this pattern has been analyzed in various ways, such as “individuals mainly trade at the opening price and institutions mainly trade at the closing price” and “due to the overreaction effect on the opening price”, etc., but no clear conclusion has been reached yet.
In order to make trading profits through only this pattern, it needs more research to find a stronger signal in the structure of the Korean stock market, where transaction taxes and transaction costs are high (30bps transaction tax). From 2002 to now, the cumulative yield of G-component is about 1200%, and it is about 0.1-0.2% on a daily basis, which is lower than the transaction tax. However, it is clear that it is advantageous to at least sell at the open price when selling stocks, and to buy at the closing price when buying stocks. And if you are a market maker or an execution broker, you can create additional revenue by using this pattern for trading.
Going back to the original story, let's take a closer look at how price movements affect subsequent price movements. A natural hypothesis would be that the size or direction of G (sign of gap return) would affect the subsequent D, or whether the size or sign of D will affect the subsequent G.
To verify this, let's decompose G into two components again.
G1: If the return of D before the gap is zero or positive. In other words, when the previous intraday return is positive, G1 is the return of the next gap. Otherwise, G1 is regarded as 0.
G2: If the return of D before the gap is negative, that is, when the previous intraday return is negative, G2 is the return of the immediately following gap, otherwise G2 is 0.
Now G = G1 + G2, every day one of G1 and G2 is zero.
If D is decomposed in the same way, D = D1 + D2, and each day either D1 or D2 becomes 0.
In simple terms, the daily rate of return P = G1 + G2 + D1 + D2. Strictly speaking, (1+P)=(1+G1)*(1+G2)*(1+D1)*(1+D2), but the difference is not so great.
If G1 and G2 have different characteristics or D1 and D2 show different characteristics, this is strong evidence that the yield sign of G and D, which is the price change immediately preceding, affects the subsequent price movement. And, in practice, this phenomenon is observed.
Decomposing of G1, G2, D1, D2 (Arithmetic Cumulative return) of KOSDAQ Composite Index (1997~2021)
Cumulative return of G1 (arithematic, daily)
Cumulative return of G2 (arithmetic, daily)
Cumulative return of D1 (arithmetic, daily)
Cumulative return of D2 (arithmetic, daily)[Daily KOSDAQ Composite Index from 1997-06-02 to 2021-12-24]
G1 event: 2752 times / cumulative return 976% / average return per trial 0.35%
G2 event: 3380 times / cumulative return 334% / average return per trial 0.10%
D1 event: 4544 times / cumulative return -1067% / average return per trial -0.23%
D2 event: 1588 times / cumulative return -161% / average return per trial -0.10%
Probability of (+) Gap return: 74%
Probability of (-) Intraday return: 55%
From the fact that the G1 and D1 performances are strong (in terms of absolute value), it can be seen that the gap return has a trend-following tendency and the intraday return has a strong reversal tendency. When a intraday return rises, the power to gap up the next day is strong, and when a gap up, the power to close with a negative intraday return is strong on that day.
The difference in the distribution of returns between G1 and G2 and the distribution of returns between D1 and D2 are quite distinct. Average return increases about 2-3 times, and volatility decreases by 15-30%. This can be good evidence that the sign of the gap return affects the subsequent intra-day return, and that the sign of the intra-day return affects the gap return the next day. (Of course, it is necessary to determine whether this is a direct causal relationship or whether it is simultaneously affected by other factors)
Markov Model
From the fact that, on average, G has an upward force and D has a downward force, the following transition flow can be considered.
Transition Flow DiagramSince G has a tendency of (+) and D has a tendency of (-), the performance of the gap up-> intra-day down-> gap up-> intra-day down pattern will be probable. In particular, even if an intraday rise occurs due to other factors in the middle of the market due to the strong (+) tendency of G1, there is a high probability of returning to the gap up through the power of G1. And it is highly probable that the chain will continue to fall, followed by a gap up -> intraday down -> gap up -> intraday down.
Orange arrows indicate transition directions with high probability and high rewardsThis can be expressed as a Markov chain as follows.
Markov chain of G1, G2, D1, D2Under this framework, each state and the transition probability between states can be estimated more precisely. The above process only shows the relationship between the gap up, the gap fall, the intraday rise, and the intraday fall, but in the end, what all quants do is add or update states (sometimes the state is hidden) and transition probabilities and payoffs according to various factors to this Markov chain. The final result of capturing non-random patterns in data through AI technology in Qraft can also be expressed in this kind of diagram. Machine learning based on this framework is also a valid method. For example, by dividing the gap rising state by the size of the gap return rather than simply sign of the gap, dividing it into multiple states (e.g. the larger the size of the gap, the greater the subsequent reaction), or increasing the number of states through other variables (e.g. when gap up and good sentiment news are combined, the follow-up reaction is large), and methods such as adding a hidden state to create a hidden Markov structure can be approached by machine learning.
More complex Markov Model (just hypothetical example)Insights from the IT bubble period
To get more insight, let's look at the data of the KOSDAQ IT bubble period. The most turbulent period in the KOSDAQ was from 1997 to 2001. The IMF crisis occurred at the end of 1997 in Korea, and this is the period between the creation of the IT bubble and its complete collapse. There was a 6-fold increase in less than a year, and there was also a drop of more than 80% in less than a year.
1999-2001 KOSDAQ Composite Index and Cumulative Returns of G1, G2, D1, D2 (arithmetic, daily)The upper graph is the KOSDAQ Composite Index, and the lower graph is decomposed cumulative returns of G1 (orange), G2 (pink), D1 (green), and D2 (blue). In this period, some unusual phenomena are observed that are different from other periods.
1) Elevation of D1 component
Interestingly, during this period, the KOSDAQ surged and there was a rise mainly through the D component, especially D1, until the second peak. This is the only time the D component rises for a substantial time since the KOSDAQ index was born.
From 1997 to 2021, the only rise in the D component was during the IT bubble.In the long section from 1997 to 2021, the component that falls the most strongly is D1, and the component that rises the most strongly is G1. However, this time was different. Although the rate of decline of the graph accumulating D returns after the COVID plunge has slowed, the IT bubble period is the first and last time when the D component itself has risen. In addition, almost all the rise of the D component during this period was made by the D1 component (continuously rising to the 2nd peak), and the return of D2 only briefly goes up in the last uptrend (from the short-term low after the 2nd peak to the 3rd peak) and then goes down again as usual.
2) Drop of G2 component
The G1 component continued its upward trend even during this turbulence period. Even when the IT bubble bursts, the G1 component continues to rise. Even in a strong down market caused by the collapse of the IT bubble, the strategy of buying at the closing price and selling at the opening price the next day when the intraday return was (+) (G1 component) was able to generate steady profits. (If transaction costs are not included, about 35% of profits are generated even during the IT bubble burst period for 9 months.) G1's steady upward trend was no exception during this period.
What is unusual is the G2 component. G2 continues to decline during this period. This is the first and last time the graph of cumulative G2 returns has declined for more than three years. (During the 2008 financial crisis, G2 fell briefly for a quarter)
3) IT bubble rises and falls
What hard-carried the rise of the IT bubble was mainly the D1 component up to the second peak. The rise towards the highest (3rd high peak) was mainly led by G1 and D2. This is unusual given that the D component usually tends to have a negative rate of return. The (+) return of D1 means that the upward trend continued even during the intraday when the market start by a gap up.
In general, D1 shows the strongest (-) tendency. Gap up is a response to information from market closing to opening the next day. Since this reaction mainly occurs overreaction, the size of the rise in the gap is usually exaggerated, and the desire for profit realization increases as a result of the rise in the gap. As a result, D1 shows a negative flow in most sections.
However, the situation was completely different during the IT bubble period. If I add my imagination, this period was a period when the energy of the market itself was boiling, many individual investors were armed with speculative spirit, and a huge number of day traders were mass-produced thanks to the introduction of online home trading system and great volatility. Rather than interpreting the information from the market close to the opening, this group of investors recognized the rise in the gap as strong positive news and bought during the intraday, and the price rise caused by this purchase stimulate other buyers. Accordingly, the D1 component was able to show a strong upward trend rather than a downward trend in a normal period. From this point of view, it is understandable that the G2 component, which shows the (+) tendency in the normal period, did not rise much. G2 means the gap return when the previous intraday return is negative. From the perspective of a day trader, there would not have been much incentive to bet on the market price of the next day when the intraday return was negative. This period was a market dominated by day trading based on intraday momentum triggers. It is no coincidence that G1 (gap up after positive intraday return) and D2 (intraday rebound after gap fall) led the final rise to the last high before the final collapse, not D1.
Although it is still a prototype, there are evidences that it is possible to predict the continuity and timing of changes in the regime of the entire market or individual stocks to some extent through analysis of the movements of G1, G2, D1, D2 or similar indicators. Significant research results are coming out that it is possible to capture the buying or selling forces. In some cases, similar results can be obtained from intraday data.
Working: Integrated Alpha with ML process
The KOSDAQ index is a composite index of all stocks, but if you go into individual stocks, more diverse and interesting patterns are found. Some of these patterns can be improved and used in real trading right away.
Single stock futures A in KRX: G1The graph above is a graph (candlestick, daily) of single stock futures in the Korean market. It can be seen that the G1 component is clearly upward sloping. Since it is an individual single stock futures, the burden of transaction costs is very low, and if the trading volume is sufficient, the strategy of buying futures at the closing price in the case of the intraday rise and liquidating the futures at the opening price the next day is possible. If calculated with compound basis, the return is 2000% (this is when transaction costs and trading volume are not taken into account). Compared to the stock being cut in half during the same period, it is a very good performance.
Single stock futures B in KRX: D2D2 component for single stock futures in another stock. There is a clear and steady downtrend, and since it is a futures, an annual return of over 30% is possible through the D2 short strategy. (If transaction costs and volume are not taken into account) It can be a good strategy to test many stocks in many markets and build a portfolio of meaningful components of G1, G2, D1, D2.
The crypto market is also a market in which a chain of strong action-reaction pattern is taking place. Perhaps because there are many speculative participants and day traders, and the proportion of individual participants who move well in behavioral economics is high, this inefficiency appears to be large. The crypto market is traded 24 hours a day, so the concept of a gap does not exist, but if you define (or train) an appropriate kernel with variables such as price fluctuations, transaction volume, and specific events, you can create a model of action -> reaction structure, such as the patterns found in this way seem to be working quite well.
Even for single stocks, if you create a variety of action-reaction structures from various viewpoints such as sectors, factors, and prices, etc, and extract meaningful patterns and calculate them, and operate them together, you can get robust performance and transaction costs can be greatly reduced. In the case of daily trading such as G1, G2, D1, and D2, transaction cost management is very important because the turnover rate is quite high. Therefore, only the netted quantity is required to be executed finally. As a result, transaction costs are significantly reduced. In particular, in cases where only closing and opening prices are traded, such as G1, G2, D1, and D2, the trading time can be precisely matched, and this effect is greater. In this netted portfolio, once again, the alpha learned from tick data level data through reinforcement learning-based AXE order execution technology is applied to reduce transaction costs once again.
There are few fields where the results of research lead to direct feedbacks and great rewards as much as the financial market. We would like to ask for the support of talented people from various fields who will unravel the secrets of price movement in the financial market through AI technology. Even if you have no experience in finance, asset management, or trading, you can achieve great results within the platform called Qraft if you have expertise in science or engineering or AI and GRIT.
Open Positions in Qraft Technologies
Previous Article: AI Asset Management Report
Qraft Technologies is
A company using AI technology to innovate the inefficiency of the asset management industry. From data processing to alpha research and portfolio order execution, we aim to solve inefficiencies at each stage of the asset management industry with AI technology. With this, we can provide a high level of alpha at a lower cost.
We automate complex financial data pre-processing, accelerate it through parallel computing, and automatically search for the alpha factor through AutoML technology in a well-established simulation environment. Using the alpha factors found in this way, a deep learning-based Deep Asset Pricing Model is created through the Strategy Factory as per the fund universe defined for a fund concept. The final portfolio made through this model is efficiently executed with the order execution engine AXE based on reinforcement learning.
Qraft Webpage: http://www.qraftec.com
Qraft Newsletter: https://www.qraftec.com/newsletter
Disclaimer
The past performance may not be indicative of future results.
This material was prepared for informational purposes and cannot be used for the purpose of soliciting the sale of financial investment products such as funds.
This document can contain the contents of the patent-pending or registered by Qraft Technologies, Inc.
