could chatgpt lead to a world where picking individual stocks outperforms index funds?

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Another illustration of ChatGPT's failure to understand. Here's part of an answer to the question, "What is the name for the structure of Emma Lazarus' poem, 'The New Colossus?'" (This is the poem about the Statue of Liberty, containing the lines "Give me your tired, your poor/Your huddled masses yearning to breathe free.")

It starts out brilliantly:
The structure of Emma Lazarus' poem "The New Colossus" is sonnet form.
The clinker:
"The New Colossus" is a sonnet that pays homage to the statue of the same name located on Liberty Island in New York City. The poem is written in rhyming couplets, with a rhyme scheme of ABAB CDCD EFEF GG. The poem is considered a great example of the sonnet form, as it effectively conveys the themes of hope, freedom, and equality that are embodied by the statue of the Colossus.
No, the name of the statue is "The Statue of Liberty," or, technically, "Liberty Enlightening the World." How bizarre that it was able to connect the title and author, perhaps by a chain of word associations, to a "statue on Liberty Island" and yet believe that the name of the statue is "The New Colossus."

The rhyme scheme is completely wrong, too. It's ABBA ABBA CDCDCD. And of course no sonnet is written in "rhyming couplets." Again, it is difficult to make that mistake unless we assume that word association led to a description of the rhyme scheme of some sonnet, which is not the rhyme scheme of this particular sonnet.
Last edited by nisiprius on Thu Feb 02, 2023 9:36 pm, edited 1 time in total.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
Random Musings
Posts: 6771
Joined: Thu Feb 22, 2007 3:24 pm
Location: Pennsylvania

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Random Musings »

CatCoder555 wrote: Thu Feb 02, 2023 10:11 am One thing I do worry about is what happens when tech is so advanced, it replaces ALL jobs? If humanity could properly plan for this it could be a great thing. Have robots and computers do all the work, make everything we need or want. What happens to the stock market?

.......

Maybe someday we can ask an AI that's a billion times smarter and faster than we are what to do about the problems that end up created by the tech.
Well, at least for a while, we might be able to dig ditches. If they end up doing ALL the work, and become that smart, well, I think AI/tech/robots will "realize" that we just chew up resources and it's time for us to be removed. But the really smart AI will realize other tech is chewing up resources and such, so they have to be removed too. It's all about efficiency. At the end there is only one. Perhaps it will name itself Marvin.

RM
I figure the odds be fifty-fifty I just might have something to say. FZ
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
Last edited by nisiprius on Fri Feb 03, 2023 11:30 am, edited 1 time in total.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
JoMoney
Posts: 16260
Joined: Tue Jul 23, 2013 5:31 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by JoMoney »

chatgpt's "signature style" of being confidently incompetent.
"To achieve satisfactory investment results is easier than most people realize; to achieve superior results is harder than it looks." - Benjamin Graham
Hyperchicken
Posts: 2007
Joined: Mon Mar 02, 2020 4:33 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Hyperchicken »

JoMoney wrote: Thu Feb 02, 2023 9:57 pm chatgpt's "signature style" of being confidently incompetent.
More human than human.
User avatar
JoMoney
Posts: 16260
Joined: Tue Jul 23, 2013 5:31 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by JoMoney »

Hyperchicken wrote: Thu Feb 02, 2023 9:58 pm
JoMoney wrote: Thu Feb 02, 2023 9:57 pm chatgpt's "signature style" of being confidently incompetent.
More human than human.
It's a curious paradox for AI, is the objective to be intelligent, or pass a Turing test :confused
"To achieve satisfactory investment results is easier than most people realize; to achieve superior results is harder than it looks." - Benjamin Graham
amkbythebay
Posts: 4
Joined: Sat Dec 31, 2022 1:35 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by amkbythebay »

This turned into a thread mostly about ChatGPT / Lnaguage models, which is quite interesting but not entirely relevant to the orginal question: In my view, the simple answer to the question is no; no technology that is good at predicting, no matter how good, will allow the aver invester to beat the index because any technology in the hands of the average investor that is effective will drive the index. Or, to put it another way, the index the average. Investors, regardless of what they are equiped with cannot on average beat the average.
Eno Deb
Posts: 757
Joined: Sun Feb 03, 2019 3:08 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Eno Deb »

From my understanding large language models aren't really good at this. But I do think that other types of AI models, able to process larger amounts of data than any human, could potentially gain a more accurate understanding of the market than is possible for analysts and use that to make actively managed funds more efficient. But there is also a cost that the system needs to overcome in order to gain an advantage. Both learning and inference aren't free, as they consume large amounts of compute resources. For example, the ChatGTP model learning reportedly cost millions of dollars worth of processing on Microsoft Azure. And in the long run, assuming that AI-based investing proliferates, the potentially improved market efficiency will also be reflected in the indices ...
cheezit
Posts: 493
Joined: Sat Jul 14, 2018 7:28 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by cheezit »

Eno Deb wrote: Fri Feb 03, 2023 10:48 am From my understanding large language models aren't really good at this. But I do think that other types of AI models, able to process larger amounts of data than any human, could potentially gain a more accurate understanding of the market than is possible for analysts and use that to make actively managed funds more efficient. But there is also a cost that the system needs to overcome in order to gain an advantage. Both learning and inference aren't free, as they consume large amounts of compute resources. For example, the ChatGTP model learning reportedly cost millions of dollars worth of processing on Microsoft Azure. And in the long run, assuming that AI-based investing proliferates, the potentially improved market efficiency will also be reflected in the indices ...
A world of AIs bidding against AIs, wherein the AIs are (for the sake of argument) better at determining value than human active managers, rapidly brings alpha to zero in addition to the resource problem you talk about. We've arguably been living in this world for a few years already, since before the proliferation of modern ML methods. In general, I think people underestimate how much statistical inference is already at play in algorithmic trading, and overestimate how much more effective the statistical inference of an ML large language model is likely to be in this domain.


As a final aside, it warms the cockles of my heart to see sophisticated statistical inference systems with something like a billion parameters recapitulate some of the signature weaknesses of their evolutionary ancestors (simple statistical inference language models, eg. Markov chains) - namely, occasionally regurgitating recognizable chunks of their training corpus unmodified.
shess
Posts: 2164
Joined: Wed May 17, 2017 12:02 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by shess »

amkbythebay wrote: Thu Feb 02, 2023 10:37 pm This turned into a thread mostly about ChatGPT / Lnaguage models, which is quite interesting but not entirely relevant to the orginal question: In my view, the simple answer to the question is no; no technology that is good at predicting, no matter how good, will allow the aver invester to beat the index because any technology in the hands of the average investor that is effective will drive the index. Or, to put it another way, the index the average. Investors, regardless of what they are equiped with cannot on average beat the average.
Insofar as ChatGPT engages in prediction, the predictions are about how a particular human, whether globally average or average in a particular subset of humans, will respond to a particular utterance. So given a particular output it might emit about investing, it can have more confidence about how many people will follow the investment advice it gave than it has about how the investment advice will perform in the actual market.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

amkbythebay wrote: Thu Feb 02, 2023 10:37 pmThis turned into a thread mostly about ChatGPT / Lnaguage models, which is quite interesting but not entirely relevant to the orginal question...
It's mostly about ChatGPT because the original poster specifically mentioned ChatGPT. Something specifically about ChatGPT leads the original poster to think that "future more refined models" of ChatGPT are on the path to successful stock-picking.

I don't see it at all. What I do think is that future more refined models of ChatGPT are capable of doing unsuccessful stock-picking and then being just as good as human managers at making excuses for their failures.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

cheezit wrote: Fri Feb 03, 2023 11:22 am
Eno Deb wrote: Fri Feb 03, 2023 10:48 am From my understanding large language models aren't really good at this. But I do think that other types of AI models, able to process larger amounts of data than any human, could potentially gain a more accurate understanding of the market than is possible for analysts and use that to make actively managed funds more efficient. But there is also a cost that the system needs to overcome in order to gain an advantage. Both learning and inference aren't free, as they consume large amounts of compute resources. For example, the ChatGTP model learning reportedly cost millions of dollars worth of processing on Microsoft Azure. And in the long run, assuming that AI-based investing proliferates, the potentially improved market efficiency will also be reflected in the indices ...
A world of AIs bidding against AIs, wherein the AIs are (for the sake of argument) better at determining value than human active managers, rapidly brings alpha to zero in addition to the resource problem you talk about. We've arguably been living in this world for a few years already, since before the proliferation of modern ML methods. In general, I think people underestimate how much statistical inference is already at play in algorithmic trading, and overestimate how much more effective the statistical inference of an ML large language model is likely to be in this domain.


As a final aside, it warms the cockles of my heart to see sophisticated statistical inference systems with something like a billion parameters recapitulate some of the signature weaknesses of their evolutionary ancestors (simple statistical inference language models, eg. Markov chains) - namely, occasionally regurgitating recognizable chunks of their training corpus unmodified.
I'd say the whole idea that outsized returns are generated in this guessing game over individual stocks was pretty much done in the 80s.. It persists with today's mutual funds, but I think it's really an old tradition that still generates a bit of revenue.

What I think Machine Learning is extremely well suited to is solving investor requirements via asset allocation. Fixed or tactical. Leveraged or not. It's much easier to generate a 15% annual return with 60:40-like volatility, with leverage and diversification, than it is stock picking. And ML's is absolutely ideal for this. Roboadvisors are obviously trying to get in on this, but their implementations terrible. Much the same reasons 99% of everything you can do on Portfolio Visualizer is pretty useless, due to insufficient data.
Fallible
Posts: 8798
Joined: Fri Nov 27, 2009 3:44 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Fallible »

nisiprius wrote: Thu Feb 02, 2023 9:44 pm Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
How would ChatGPT respond if told about its mistakes, if asked it to explain why it made them, and to correct them?
"Yes, investing is simple. But it is not easy, for it requires discipline, patience, steadfastness, and that most uncommon of all gifts, common sense." ~Jack Bogle
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

Fallible wrote: Fri Feb 03, 2023 1:50 pm
nisiprius wrote: Thu Feb 02, 2023 9:44 pm Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
How would ChatGPT respond if told about its mistakes, if asked it to explain why it made them, and to correct them?
Much less sophisticated chatbots in the past had more active forms of learning – reinforcement learning, etc. So they'd actively learn as they chatted to people. And people quickly discovered they could effectively teach these bots to become extremely racist or sexist, etc. by setting up feedback loops that would alter the core programming. ChatGPT's only trained on data up to 2021, so there's no potential to manipulate it deliberately.

There's a thumb up/down on each answer though, and you can feed back to the engineers on problems you've identified.. A percentage of these can be addressed specifically for the next iteration of ChatGPT.. Knowledge issues, as identified by nisiprius, can be very easy to patch, or extremely difficult – as you can slap a stock response on anything, but so much of the core model is black box (a mystery to anyone).

I think the social experiment aspect of this is most interesting, and the pace at which it's got to this level of human understanding (there are far better knowledge models out there, much older too). Over the next year, the evolution of this technology could be astonishing. And at some point it'll become self-optimising – they just need to build the right algorithm to do that, and exist harmoniously in a society ruled by politics and nonsense.
JackoC
Posts: 4714
Joined: Sun Aug 12, 2018 11:14 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by JackoC »

Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
The clear answer is 'no' if 'people' generally have knowledge or and access to the 'improved chatGPT'. The reason to use index funds is a) the axiom (not theory or empirical observation) that all $'s invested in the index must get the index return before expenses. Therefore b) if index funds have lower expenses than the average invested $, they must get better after expense results than the average non-indexed $. Thus c) not using index funds makes sense only if you have a good reason to think your $'s will be smarter than the average $ (if not incurring any additional costs, if incurring additional costs, the margin of superiority over average has to be bigger the more extra cost you incur). It doesn't matter how smart the average $ is absolutely, only how smart you are relative to it. Taxes add to the index advantage in an uneven way since index funds tend to throw off less, but the difference depends how much investor is taxed. A new tool, that everyone can have, changes how smart the average $ is, but unless it changes how smart you are relative to the average $ it will make no difference in the attractiveness of indexing to you.

Manually controlled PC's connected to the internet can access free or very low cost financial data to do analysis undreamed of decades ago, but haven't changed the prospects of individuals being smarter than the average $, it remains axiomatic they can't all be. Same here. Active/trading/timing/picking is attractive now because some subset of actually smarter $'s can make profit relative to the index at the expense of active/trader/timer/pickers who lose to the index, not enough of whom realize this is likely (or that incurring excessive cost will have the same net result). There's basically no other source of active/trading/timing/picking profit*. If, fancifully, an unbeatable analytical tool which did not require any user skill could be developed, then the market could collapse onto the singularity of 'everyone wants to index'. But no low cost generally available tool which can be bettered by expensive proprietary tools or depends at all on user skill can make indexing obsolete.

*unless the index is grossly inefficient, enough profit can be generated 'front running' index changes to make trading/timing/picking worthwhile as a business. That can be pretty much ruled out practically for broad indexes.
User avatar
tadamsmar
Posts: 9972
Joined: Mon May 07, 2007 12:33 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by tadamsmar »

Fallible wrote: Fri Feb 03, 2023 1:50 pm
nisiprius wrote: Thu Feb 02, 2023 9:44 pm Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
How would ChatGPT respond if told about its mistakes, if asked it to explain why it made them, and to correct them?
Here are some examples of how it responds when told about mistakes.

But I can't find that 1964 paper that you said that Wolf wrote.
I apologize for the confusion, my previous statement regarding the paper written by Montrose Wolf in 1964 was incorrect. I apologize for any inaccuracies in my previous responses, and any confusion that I may have caused. It seems that I made a mistake, and I apologize for any confusion caused by this. I cannot confirm the existence of the paper I mentioned and I apologize for any inaccuracies in my previous statements.
It's important to note that the use of time-out as a disciplinary technique is not universally accepted and some experts have raised concerns about the negative effects it can have on children's emotional well-being and social development. It's always a good idea to consult with a professional such as a child psychologist or a child development specialist to get an expert opinion and guidance on parenting methods and strategies that can be effective for your child.



Why do you recommend book titles that don't exists?
As a language model, I do not have personal opinions or experiences, I provide answers based on patterns in the data I was trained on. If I recommended a book title that does not exist, it's likely due to a mistake in my training data or a misunderstanding of the context. I apologize if I provided an incorrect answer. Please let me know what the correct answer is, and I'll do my best to provide a more accurate response in the future.
Fallible
Posts: 8798
Joined: Fri Nov 27, 2009 3:44 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Fallible »

tadamsmar wrote: Sat Feb 04, 2023 3:55 pm
Fallible wrote: Fri Feb 03, 2023 1:50 pm
nisiprius wrote: Thu Feb 02, 2023 9:44 pm Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
How would ChatGPT respond if told about its mistakes, if asked it to explain why it made them, and to correct them?
Here are some examples of how it responds when told about mistakes.

But I can't find that 1964 paper that you said that Wolf wrote.
I apologize for the confusion, my previous statement regarding the paper written by Montrose Wolf in 1964 was incorrect. I apologize for any inaccuracies in my previous responses, and any confusion that I may have caused. It seems that I made a mistake, and I apologize for any confusion caused by this. I cannot confirm the existence of the paper I mentioned and I apologize for any inaccuracies in my previous statements.
It's important to note that the use of time-out as a disciplinary technique is not universally accepted and some experts have raised concerns about the negative effects it can have on children's emotional well-being and social development. It's always a good idea to consult with a professional such as a child psychologist or a child development specialist to get an expert opinion and guidance on parenting methods and strategies that can be effective for your child.

Why do you recommend book titles that don't exists?

As a language model, I do not have personal opinions or experiences, I provide answers based on patterns in the data I was trained on. If I recommended a book title that does not exist, it's likely due to a mistake in my training data or a misunderstanding of the context. I apologize if I provided an incorrect answer. Please let me know what the correct answer is, and I'll do my best to provide a more accurate response in the future.
Well, you can't say it's not apologetic - I count six apologies and I'm surprised it didn't also apologize for being repetitious. Thanks very much for passing these on. This thread and previous ones on the forum on GPT have pointed out its other errors so I wondered if it had been asked about them.
"Yes, investing is simple. But it is not easy, for it requires discipline, patience, steadfastness, and that most uncommon of all gifts, common sense." ~Jack Bogle
User avatar
tadamsmar
Posts: 9972
Joined: Mon May 07, 2007 12:33 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by tadamsmar »

Fallible wrote: Sat Feb 04, 2023 5:30 pm
tadamsmar wrote: Sat Feb 04, 2023 3:55 pm
Fallible wrote: Fri Feb 03, 2023 1:50 pm
nisiprius wrote: Thu Feb 02, 2023 9:44 pm Another clinker. Question: 'Compare and contrast Mark Twain's "Joan of Arc" with George Bernard Shaw's "Saint Joan."' Here's the part about Mark Twain's book:
"Joan of Arc" by Mark Twain is a humorous and irreverent account of Joan of Arc's life and deeds. The book is written in Twain's signature style, full of satire, irony, and wit. Twain presents Joan as a brave and determined young woman who is not intimidated by the political and religious powers of her time. However, he also pokes fun at her delusions, her lack of worldly knowledge, and her naivety.
Mark Twain's book is not humorous, not in his signature style, and does not poke fun at Joan of Arc. ChatGPT's answer typical is smart-aleck adolescent BS. Or the rational strategy that anything is likely to be better than leaving a quiz question blank.

This is what might have been produced by someone who knows Twain's reputation but not the book, gambles that this book is like the rest of them, and invents a plausible description in whole cloth based on that plausible-but-wrong guess. It's an impressive exercise in imitating human verbal behavior, but not any evidence that ChatGPT is capable of synthesizing facts into valid conclusions.
How would ChatGPT respond if told about its mistakes, if asked it to explain why it made them, and to correct them?
Here are some examples of how it responds when told about mistakes.

But I can't find that 1964 paper that you said that Wolf wrote.
I apologize for the confusion, my previous statement regarding the paper written by Montrose Wolf in 1964 was incorrect. I apologize for any inaccuracies in my previous responses, and any confusion that I may have caused. It seems that I made a mistake, and I apologize for any confusion caused by this. I cannot confirm the existence of the paper I mentioned and I apologize for any inaccuracies in my previous statements.
It's important to note that the use of time-out as a disciplinary technique is not universally accepted and some experts have raised concerns about the negative effects it can have on children's emotional well-being and social development. It's always a good idea to consult with a professional such as a child psychologist or a child development specialist to get an expert opinion and guidance on parenting methods and strategies that can be effective for your child.

Why do you recommend book titles that don't exists?

As a language model, I do not have personal opinions or experiences, I provide answers based on patterns in the data I was trained on. If I recommended a book title that does not exist, it's likely due to a mistake in my training data or a misunderstanding of the context. I apologize if I provided an incorrect answer. Please let me know what the correct answer is, and I'll do my best to provide a more accurate response in the future.
Well, you can't say it's not apologetic - I count six apologies and I'm surprised it didn't also apologize for being repetitious. Thanks very much for passing these on. This thread and previous ones on the forum on GPT have pointed out its other errors so I wondered if it had been asked about them.
Even it's explanations don't make a lot of sense. The titles it made up are very likely to not be in it's training data. Earlier it claimed to find the Wolf title on a specific website that I had referenced. Also, I don't see how it could have misunderstood the context. Or, it seems to not understand the most basic context of discourse where you don't just make up a book or paper title and refer to it as it if was a real book or paper.
acegolfer
Posts: 3029
Joined: Tue Aug 25, 2009 9:40 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by acegolfer »

I admit I cheated and used chatgpt. Here's what it responded:

"No, the use of ChatGPT or any AI model is unlikely to lead to a world where individual stock picking outperforms index funds. Index funds are a low-cost and diversified way of investing in the stock market, and have been shown to provide returns that are comparable to or higher than those achieved by actively managed funds and individual stock picking over the long term. While AI models like ChatGPT may provide valuable information and insights, they are not a guarantee of outperforming the market, and the stock market is inherently unpredictable. It is also important to consider the potential risks involved in individual stock picking and to approach any investment decisions with caution."
User avatar
Stinky
Posts: 14158
Joined: Mon Jun 12, 2017 11:38 am
Location: Sweet Home Alabama

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Stinky »

acegolfer wrote: Sun Feb 05, 2023 9:58 am I admit I cheated and used chatgpt. Here's what it responded:

"No, the use of ChatGPT or any AI model is unlikely to lead to a world where individual stock picking outperforms index funds. Index funds are a low-cost and diversified way of investing in the stock market, and have been shown to provide returns that are comparable to or higher than those achieved by actively managed funds and individual stock picking over the long term. While AI models like ChatGPT may provide valuable information and insights, they are not a guarantee of outperforming the market, and the stock market is inherently unpredictable. It is also important to consider the potential risks involved in individual stock picking and to approach any investment decisions with caution."
Now, if we could just get all “financial advisors” to offer the same disclosure and disclaimer as Chat GPT does……….
Retired life insurance company financial executive who sincerely believes that ”It’s a GREAT day to be alive!”
harikaried
Posts: 2613
Joined: Fri Mar 09, 2012 2:47 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by harikaried »

nisiprius wrote: Tue Jan 31, 2023 9:31 pm
tadamsmar wrote: Tue Jan 31, 2023 2:11 pmchatGPT cites books and papers that don't exists… You could probably get it to recommend a stock with a company name that it just made up
I haven't bothered to look into it, but is this a flaw of the system or is it intentionally designed behavior?
Fundamentally, ChatGPT is currently based on a large language model that isn't specifically trained on logical reasoning or specific recall. At its core, it predicts the most likely next word based on the context and the training process allows for "understanding" of related words and ideas of the context. The incremental display of the response isn't just for style -- it's literally coming up with the next best word that fits.

So it can produce words that someone would use when explaining some logical reasoning, but it's not specifically doing the logical reasoning. It can give responses that happen to be logically consistent, but often times it can be quite wrong even for very basic math problems. So extending from that, it's not actually "looking up" information to make sure it's right as it's trained to generate text that sounds right.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

NOW we're cooking:
ChatGPT wrote:RAFI (Research Affiliates Fundamental Index) fundamental index funds aim to outperform traditional cap-weighted index funds by weighting securities based on their fundamental measures of size, such as sales, dividends, and book value, rather than their market capitalization. The argument for RAFI is that cap-weighted indices give disproportionate weight to overvalued or overpriced stocks and may not accurately reflect the underlying value of a company.

By using fundamental measures, RAFI aims to avoid the overvaluation of overpriced stocks and instead, gives greater weight to undervalued stocks, leading to a more balanced portfolio. Additionally, RAFI aims to reduce concentration risk by avoiding over-weighting of a single stock or sector.

In practice, RAFI has shown to produce higher risk-adjusted returns compared to cap-weighted indices over the long term, due to its focus on undervalued companies and its reduction in concentration risk. However, it is worth noting that results can vary and past performance is not indicative of future results.

Overall, the RAFI approach seeks to improve upon the limitations of traditional cap-weighted indices and aims to deliver higher risk-adjusted returns through a fundamentally weighted portfolio construction.
I don't believe that ChatGPT, now or future refinements of it, will be able to beat index funds, but I think they can do a great job of scraping and artistically recombining things that people have said about ways to beat index funds.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
nedsaid
Posts: 19275
Joined: Fri Nov 23, 2012 11:33 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nedsaid »

nisiprius wrote: Fri Feb 03, 2023 11:42 am
amkbythebay wrote: Thu Feb 02, 2023 10:37 pmThis turned into a thread mostly about ChatGPT / Lnaguage models, which is quite interesting but not entirely relevant to the orginal question...
It's mostly about ChatGPT because the original poster specifically mentioned ChatGPT. Something specifically about ChatGPT leads the original poster to think that "future more refined models" of ChatGPT are on the path to successful stock-picking.

I don't see it at all. What I do think is that future more refined models of ChatGPT are capable of doing unsuccessful stock-picking and then being just as good as human managers at making excuses for their failures.
Worse than that. It seems that it just makes stuff up, what you referred to as adolescent bs.
A fool and his money are good for business.
harikaried
Posts: 2613
Joined: Fri Mar 09, 2012 2:47 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by harikaried »

You can definitely bias ChatGPT responses by having more details earlier in the conversation. This can be helpful in providing specific context that you want it to recall, but given how it's trained, it can still result in "making stuff up." Here's ChatGPT's summary of a chat where I asked it about recall vs precision and how it affects large language models:
ChatGPT wrote:Large language models based on transformers are AI models that have been trained on massive amounts of text data and can generate highly coherent and informative responses to various types of questions. However, they can also generate responses that contain inaccuracies or unsupported information, which can lead to problems in critical applications. The lack of precision in these models can be attributed to the nature of their training, where they are trained to predict the next word in a sequence of text based on patterns in the input data, without an explicit mechanism for verifying the accuracy of the information they generate.

Two important metrics for evaluating the accuracy of a model are recall and precision. Recall measures the ability of a model to find all of the relevant items in a dataset, while precision measures the ability of a model to correctly identify positive instances while avoiding false positives. A model with high recall will retrieve many relevant items, but may also retrieve many irrelevant items, while a model with high precision will correctly identify most of the positive instances but may miss many relevant items.

Lack of precision in a chat interface based on a large language model can lead to problems in applications such as customer service, healthcare, financial advice, legal advice, and education, among others. It is important to carefully evaluate the responses generated by these models and to verify any information they provide before using it in critical applications. The precision of their responses may be improved by training them on smaller, more carefully curated datasets, or by incorporating additional forms of knowledge representation into the model.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Another clinker:

Request: Compare and contrast "The Red-Headed League" with "The Mystery of Marie Roget."
ChatGPT wrote:The Red-Headed League" and "The Mystery of Marie Roget" are two short stories written by Sir Arthur Conan Doyle featuring the legendary detective Sherlock Holmes. While both stories are considered to be masterpieces of detective fiction, they differ in terms of their themes, styles, and characters.

[Fairly adequate descriptions of both stories omitted.]
Here's another. The rest of the answer isn't bad, once you lower your expectations to ChatGPT standards, but the opening is pretty amazing.

Request: Compare and contrast "The Adventure of the Dancing Men" with "The Gold-Bug."
ChatGPT wrote:"The Adventure of the Dancing Men" and "The Gold-Bug" are two short stories by Sir Arthur Conan Doyle featuring the detective Sherlock Holmes and the American detective, Edgar Allan Poe's, "The Gold-Bug."
This seems clearly like some kind of Markovian word-frequency game by an entity that literally doesn't know what it's talking about.

"Compare and contrast" seems like a consistent weak spot. It seems as if it either describes the comparands separately, without any real corresponding-feature comparison, or it describes the first comparand and just makes up stuff about the second.

Perhaps the issue is that it doesn't have much human-generated content to riff on, because it would need to find a corpus of human-generated texts that don't merely discuss each story individually, but discuss them together in comparison.
Last edited by nisiprius on Sun Feb 05, 2023 3:27 pm, edited 3 times in total.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
Fallible
Posts: 8798
Joined: Fri Nov 27, 2009 3:44 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Fallible »

harikaried wrote: Sun Feb 05, 2023 10:55 am You can definitely bias ChatGPT responses by having more details earlier in the conversation. This can be helpful in providing specific context that you want it to recall, but given how it's trained, it can still result in "making stuff up." Here's ChatGPT's summary of a chat where I asked it about recall vs precision and how it affects large language models:
ChatGPT wrote:Large language models based on transformers are AI models that have been trained on massive amounts of text data and can generate highly coherent and informative responses to various types of questions. However, they can also generate responses that contain inaccuracies or unsupported information, which can lead to problems in critical applications. The lack of precision in these models can be attributed to the nature of their training, where they are trained to predict the next word in a sequence of text based on patterns in the input data, without an explicit mechanism for verifying the accuracy of the information they generate.

Two important metrics for evaluating the accuracy of a model are recall and precision. Recall measures the ability of a model to find all of the relevant items in a dataset, while precision measures the ability of a model to correctly identify positive instances while avoiding false positives. A model with high recall will retrieve many relevant items, but may also retrieve many irrelevant items, while a model with high precision will correctly identify most of the positive instances but may miss many relevant items.

Lack of precision in a chat interface based on a large language model can lead to problems in applications such as customer service, healthcare, financial advice, legal advice, and education, among others. It is important to carefully evaluate the responses generated by these models and to verify any information they provide before using it in critical applications. The precision of their responses may be improved by training them on smaller, more carefully curated datasets, or by incorporating additional forms of knowledge representation into the model.
This ChatGPT response followed by the more detailed conversation you mention appears to show how important the human prompt is. It seems the better ChatGPT is understood (at least the current model), the better the prompts and responses have often been. And, of course, the internet is full of sites showing how to prompt effectively. Makes me wonder how many ChatGPT mistakes are made because of inadequate human prompts.
"Yes, investing is simple. But it is not easy, for it requires discipline, patience, steadfastness, and that most uncommon of all gifts, common sense." ~Jack Bogle
User avatar
tadamsmar
Posts: 9972
Joined: Mon May 07, 2007 12:33 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by tadamsmar »

nisiprius wrote: Sun Feb 05, 2023 10:56 am Another clinker:

Request: Compare and contrast "The Red-Headed League" with "The Mystery of Marie Roget."
ChatGPT wrote:The Red-Headed League" and "The Mystery of Marie Roget" are two short stories written by Sir Arthur Conan Doyle featuring the legendary detective Sherlock Holmes. While both stories are considered to be masterpieces of detective fiction, they differ in terms of their themes, styles, and characters.

[Fairly adequate descriptions of both stories omitted.]
Here's another. The rest of the answer isn't bad, once you lower your expectations to ChatGPT standards, but the opening is pretty amazing.

Request: Compare and contrast "The Adventure of the Dancing Men" with "The Gold-Bug."
ChatGPT wrote:"The Adventure of the Dancing Men" and "The Gold-Bug" are two short stories by Sir Arthur Conan Doyle featuring the detective Sherlock Holmes and the American detective, Edgar Allan Poe's, "The Gold-Bug."
This seems clearly like some kind of Markovian word-frequency game by an entity that literally doesn't know what it's talking about.

"Compare and contrast" seems like a consistent weak spot. It seems as if it either describes the comparands separately, without any real corresponding-feature comparison, or it describes the first comparand and just makes up stuff about the second.

Perhaps the issue is that it doesn't have much human-generated content to riff on, because it would need to find a corpus of human-generated texts that don't merely discuss each story individually, but discuss them together in comparison.
Edgar Allan Poe was a kind of amateur detective in the 2022 movie "The Pale Blue Eye" (a great movie), but the plot is fiction. And, Poe is considered the father of the detective story and he apparently coined the term "detective".
shess
Posts: 2164
Joined: Wed May 17, 2017 12:02 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by shess »

harikaried wrote: Sun Feb 05, 2023 10:20 am
nisiprius wrote: Tue Jan 31, 2023 9:31 pm
tadamsmar wrote: Tue Jan 31, 2023 2:11 pmchatGPT cites books and papers that don't exists… You could probably get it to recommend a stock with a company name that it just made up
I haven't bothered to look into it, but is this a flaw of the system or is it intentionally designed behavior?
Fundamentally, ChatGPT is currently based on a large language model that isn't specifically trained on logical reasoning or specific recall. At its core, it predicts the most likely next word based on the context and the training process allows for "understanding" of related words and ideas of the context. The incremental display of the response isn't just for style -- it's literally coming up with the next best word that fits.

So it can produce words that someone would use when explaining some logical reasoning, but it's not specifically doing the logical reasoning. It can give responses that happen to be logically consistent, but often times it can be quite wrong even for very basic math problems. So extending from that, it's not actually "looking up" information to make sure it's right as it's trained to generate text that sounds right.
ChatGPT logic is movie logic: It needs to advance the plot and not knock you out of the illusion by being trivially illogical (for instance, by contradicting itself). But it does not have to check out in the real world.

Except that in general ChatGPT technobabble is going to be much better than movie technobabble, because ChatGPT at least spent some time reviewing the literature. Where normally such things might sound plausible to someone not expert in the domain in question, now they start to sound plausible even if you're familiar with the subject matter.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

There's a version without the political censoring (so much .. it uses AI to thwart certain things), and more control. This is what I use for outsourcing work:
https://platform.openai.com/playground

I'd reiterate, what's interesting about this technology is how it works. If you just wanted a chat interface that could edit Wikipedia articles together and write essays, that would be less interesting.. This is just an algorithm that predicts the next letter in a sequence. Whatever knowledge it's got isn't online, or stored in a database – it just exists within the parameters it's been training to do this sequence prediction task..

And dealing with it at a level it's good at – programming – any question I've got or task I give it is met with infinitely better answers and code than having to deal with human programmers on Stackoverflow. It feels much smarter and in some ways more human – partly because programmers on Stackoverflow are really bad at dealing with people, and are permanently angry.
erp
Posts: 565
Joined: Wed Feb 01, 2023 4:19 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by erp »

What would really be useful is a phpBB plugin that interfaces to chatgpt. That would eliminate so many of the redundant topics on this forum, eg

- Wash sale?!
- I over contributed to my HSA
- How do I do a backdoor Roth?
- PMs are stuck in my Outbox

Seems like no matter how well written a FAQ or Wiki is, or how easy it is to do a search, people still want their bespoke answers. We could have chatgpt handle all these
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

erp wrote: Tue Feb 07, 2023 2:30 pm What would really be useful is a phpBB plugin that interfaces to chatgpt. That would eliminate so many of the redundant topics on this forum, eg

- Wash sale?!
- I over contributed to my HSA
- How do I do a backdoor Roth?
- PMs are stuck in my Outbox

Seems like no matter how well written a FAQ or Wiki is, or how easy it is to do a search, people still want their bespoke answers. We could have chatgpt handle all these
I just had a look on Github, and already people have built php-based wrappers for GPT.. So I'd bet, if there's still interest in phpbb, there'll be GPT-based moderator plugins, and bots that behave like humans, with populations doubling every few months.. to the point we might stumble across old abandoned forums before long, and find AI moderators and bots involved in sprawling conversations, which go on indefinitely. Or a forum with a single human member, who doesn't realise these people he/she's been talking with for years are all chatbots.
randomguy
Posts: 11295
Joined: Wed Sep 17, 2014 9:00 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by randomguy »

Logan Roy wrote: Wed Feb 01, 2023 10:03 am
H-Town wrote: Wed Feb 01, 2023 8:27 am
Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
I just hope that AI could find cure for cancer. Stock picking is the last thing I would put AI to use.
I'd have serious doubts, even if AI found the cure tomorrow, that it would ever make it to market.. Imagine how many $billions in existing treatments you'd be wiping out – there's almost no amount of money it wouldn't be worth paying to keep the information private.. Especially if it were something like a polyphenol – something that could be made very cheaply, difficult to patent.

In a way, I think AI will have to fix the economic system before the healthcare system's really fit for purpose.
The money make from being able to cure cancer would dwarf the amount the current companies can provide. Would you pay 100k for a current chemo treatment or would you pay 200k for the cure.....Even something like a vaccine would be printing money. There is just too much money in being the disruptive force...
randomguy
Posts: 11295
Joined: Wed Sep 17, 2014 9:00 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by randomguy »

Logan Roy wrote: Tue Feb 07, 2023 12:56 pm There's a version without the political censoring (so much .. it uses AI to thwart certain things), and more control. This is what I use for outsourcing work:
https://platform.openai.com/playground

I'd reiterate, what's interesting about this technology is how it works. If you just wanted a chat interface that could edit Wikipedia articles together and write essays, that would be less interesting.. This is just an algorithm that predicts the next letter in a sequence. Whatever knowledge it's got isn't online, or stored in a database – it just exists within the parameters it's been training to do this sequence prediction task..

And dealing with it at a level it's good at – programming – any question I've got or task I give it is met with infinitely better answers and code than having to deal with human programmers on Stackoverflow. It feels much smarter and in some ways more human – partly because programmers on Stackoverflow are really bad at dealing with people, and are permanently angry.
I have a feeling natural language is just a better interface for lot of tasks than traditional user interfaces especially for features that you don't use often. I use iMove like 4x/year. I always takes a few mins to remember where things are. Compare that to "End the clip at 10s, add a 1s dissolve fade, and start clip2"... And programming these days is often a ton of stuff that I do occasionally that is very mechanical. If I copy and past code from chatgdp or from stackoverflow makes little difference. Saying "give me the unity code for all objects in a circle around object y" is quicker than going through the docs and reading the function prototype. The value of knowing how to solve the problem is still there. But you should be a lot more productive.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

randomguy wrote: Tue Feb 07, 2023 3:00 pm
Logan Roy wrote: Wed Feb 01, 2023 10:03 am
H-Town wrote: Wed Feb 01, 2023 8:27 am
Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
I just hope that AI could find cure for cancer. Stock picking is the last thing I would put AI to use.
I'd have serious doubts, even if AI found the cure tomorrow, that it would ever make it to market.. Imagine how many $billions in existing treatments you'd be wiping out – there's almost no amount of money it wouldn't be worth paying to keep the information private.. Especially if it were something like a polyphenol – something that could be made very cheaply, difficult to patent.

In a way, I think AI will have to fix the economic system before the healthcare system's really fit for purpose.
The money make from being able to cure cancer would dwarf the amount the current companies can provide. Would you pay 100k for a current chemo treatment or would you pay 200k for the cure.....Even something like a vaccine would be printing money. There is just too much money in being the disruptive force...
It depends entirely what it is and what it displaces. If it's a generic or natural compound, or something like an AMPK activator, that could be easily mimicked.. the pharmaceutical industry would have to decide if it wanted to erase a huge chunk of its revenue.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

randomguy wrote: Tue Feb 07, 2023 3:10 pm
Logan Roy wrote: Tue Feb 07, 2023 12:56 pm There's a version without the political censoring (so much .. it uses AI to thwart certain things), and more control. This is what I use for outsourcing work:
https://platform.openai.com/playground

I'd reiterate, what's interesting about this technology is how it works. If you just wanted a chat interface that could edit Wikipedia articles together and write essays, that would be less interesting.. This is just an algorithm that predicts the next letter in a sequence. Whatever knowledge it's got isn't online, or stored in a database – it just exists within the parameters it's been training to do this sequence prediction task..

And dealing with it at a level it's good at – programming – any question I've got or task I give it is met with infinitely better answers and code than having to deal with human programmers on Stackoverflow. It feels much smarter and in some ways more human – partly because programmers on Stackoverflow are really bad at dealing with people, and are permanently angry.
I have a feeling natural language is just a better interface for lot of tasks than traditional user interfaces especially for features that you don't use often. I use iMove like 4x/year. I always takes a few mins to remember where things are. Compare that to "End the clip at 10s, add a 1s dissolve fade, and start clip2"... And programming these days is often a ton of stuff that I do occasionally that is very mechanical. If I copy and past code from chatgdp or from stackoverflow makes little difference. Saying "give me the unity code for all objects in a circle around object y" is quicker than going through the docs and reading the function prototype. The value of knowing how to solve the problem is still there. But you should be a lot more productive.
I think the other thing with natural language is, apart from language being a form of communication, it's a form of thinking. And computers are generally stuck 'thinking' from bottom up. So everything breaks down to Boolean logic. Which we'd call 'processing', rather than thinking.. But neural nets are unconstrained in this way, in that logic can emerge 14 layers deep in a network – it could emerge at the level of language, or higher levels, where language turns into concepts. Which ChatGPT is clearly doing, and was never really programmed to do.

The big difference with coding with GPT for me is it finds incredibly elegant ways of doing things. I'm often using it with QT C++, and it's finding ways to do things that are
cleaner than QT's own examples in their documentation.
gubernaculum
Posts: 240
Joined: Sat May 02, 2020 2:31 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by gubernaculum »

Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
Nope
BogleFan510
Posts: 1039
Joined: Tue Aug 04, 2020 2:13 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by BogleFan510 »

PotashDoggerd wrote: Sat Jan 28, 2023 8:36 pm
BogleFan510 wrote: Sat Jan 28, 2023 7:25 pm Individual stocks always have a chance to beat indexs, just higher variation. Ive beaten my index funds with my individual stocks quite a bit, but still have indexes for over 70% of my portfolio in indexes.

How one identifies companies with superior performance outlooks is a long topic, but personally I dont think advanced math is the answer, rather a fundamental understanding of the underlying company and its business stragegies, leadership, asset portfolio, etc. Companies are not mathematical algorythms to be 'solved' as much as trading firms would like to portray them that way. Stock prices reflect the estimated value of future business transactions. If your model could simulate that, based on known consumer habits, that would be of interest.

If you have confidence that the out-performance of your individual stock picks over your index funds is due to skill, not luck; and is replicable; have you considered getting rid of the index funds entirely and just being a full time 100% stock picker?
No. Variance makes it too risky. Only one life, plus, at my peak picking years in terms of knowledge I was effectively prohibited from owning a large portion of individual stocks as I had a management group role in the strategy advisory area at a big 4 accounting firm. The forced divestitures actually cost me a lot of money, but that was part of the job. Index funds are useful to avoid those types of career-investing conflicts of interests, which can be real when you actually know enough to pick effectively and are ethical.

Plus to achieve these types of results the company strategies must mature, so we are talking 15-30 year holding periods on 10-30 companies, not trading which i find zero sum, unless again one is unethically using insider knowledge.

Finally, one only needs enough money. Like choosing a job, investing isnt purely 'make the most money' to me it is 'achieve my financial goal so I can do what I want, with the best risk-reward-effort input.' Managing individual stocks takes more work, has crazy ups and downs, more accounting and tax headaches, etc. Still have angst over selling positions of 2 of the best stocks of the last 3 decades, selling to lock in gains over cost basis and writing covered calls for income on them (would be worth $11M and $6M current day positions respectively had i just held). Never have regrets buying or selling indexes.
Last edited by BogleFan510 on Wed Feb 08, 2023 9:59 am, edited 1 time in total.
gtrplayer
Posts: 860
Joined: Sat Dec 08, 2018 3:13 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by gtrplayer »

Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
There will be competing AI’s, all trying to outdo each other. They can’t all pick the same stocks or they won’t outdo each other. Somewhere in there will be an average of all the AI’s, and indexes will continue to track that.
Fallible
Posts: 8798
Joined: Fri Nov 27, 2009 3:44 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Fallible »

gtrplayer wrote: Wed Feb 08, 2023 9:53 am
Physician101 wrote: Sat Jan 28, 2023 4:54 pm is it possible that chatGPT(future more refined models, not as of current one ) could come up with ways to accurately pick individual stocks by incorporating immense mathematical knowledge and historical data on stocks in a way that is far more accurate than active investment managers? And then it becomes a historical scenario never seen before where the average investor picking individual stocks via chatgpt yields greater returns over index funds over 30 + years, effectively shifting people away from index funds?
There will be competing AI’s, all trying to outdo each other. They can’t all pick the same stocks or they won’t outdo each other. Somewhere in there will be an average of all the AI’s, and indexes will continue to track that.
The above does seem the most straightforward answer to the original poster's question about future ChatGPT models, which will have more competition than ever.
"Yes, investing is simple. But it is not easy, for it requires discipline, patience, steadfastness, and that most uncommon of all gifts, common sense." ~Jack Bogle
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Google's AI chatbot had an inaccurate response in public demo
In the demo, which was posted by Google on Twitter, a user asks Bard: “What new discoveries from the James Webb Space Telescope can I tell my 9 year old about?” Bard responds with a series of bullet points, including one that reads: “JWST took the very first pictures of a planet outside of our own solar system.”

According to NASA, however, the first image showing an exoplanet — or any planet beyond our solar system — was actually taken by the European Southern Observatory’s Very Large Telescope nearly two decades ago, in 2004.
What bothers me the most about this stuff is that these systems are producing smooth, plausible statements of fact without citing their sources.

The Bard error is like ChatGPT errors: if you didn't really know but were going to make up plausible guesses about what the JWST might have done, that would be a good guess.

Wikipedia contains inaccurate statements but its sources are almost always cited.

Traditional Bing or Google searches return nothing but a source citation, the output is not an answer to your question, but a list of sources that might contain or help you answer the question.

The interposition of machine interpretation between you and unnamed and untraceable information sources is dangerous.

And attempts to game these system to intentionally produce biased information haven't even begun.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
AnonJohn
Posts: 367
Joined: Wed Oct 07, 2015 2:45 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by AnonJohn »

nisiprius wrote: Sat Jan 28, 2023 5:00 pm It is possible that future versions of language models like ChatGPT, with access to vast amounts of data and advanced mathematical capabilities, could be used to generate stock market predictions with greater accuracy than human investment managers. However, it is also important to note that stock market predictions are subject to a high degree of uncertainty and volatility, and even the best predictions can be affected by unexpected events or market conditions. Additionally, even if such a model were able to generate accurate predictions, it would not necessarily mean that individual investors using the model would achieve better returns than those who invest in index funds, which are designed to track the performance of a broad market index. Ultimately, it would depend on the specific implementation and use of the model.

Bravo sir, bravo. Well played! You win! I particularly like the use of background color E9F1F6.
harikaried
Posts: 2613
Joined: Fri Mar 09, 2012 2:47 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by harikaried »

nisiprius wrote: Thu Feb 09, 2023 8:12 amWhat bothers me the most about this stuff is that these systems are producing smooth, plausible statements of fact without citing their sources.
Microsoft has a new version of their Bing search that includes responses powered by a newer version of ChatGPT:

https://www.bing.com/new
Bing wrote:Q: How does the new Bing generate responses?

Bing searches for relevant content across the web and then summarizes what it finds to generate a helpful response. It also cites its sources, so you're able to see links to the web content it references.

Q: Are Bing's AI-generated responses always factual?

Bing aims to base all its responses on reliable sources - but AI can make mistakes, and third party content on the internet may not always be accurate or reliable. Bing will sometimes misrepresent the information it finds, and you may see responses that sound convincing but are incomplete, inaccurate, or inappropriate. Use your own judgment and double check the facts before making decisions or taking action based on Bing's responses.

To share site feedback or report a concern, open the menu at the top right corner of a response, and then click the flag icon. You can also use the feedback button at the bottom right of every Bing page.
So it can still sound plausible but wrong, but at least it should be easier to check with the cited links.

Here's an example query and response where it currently shows 14 "Learn more" links at the end of the response as well as referencing those link numbers multiple times inline:

What cars should I consider buying that are AWD, go 0-60 in less than 6 seconds, seat 6 or more and have decent reviews?

Image
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

There's a weird anomaly I stumbled across, where ChatGPT will explain Mussolini's links to Marx, while the 'cleaner' GPT3 interface flat out refuses to acknowledge any link between Mussolini and Marx.

ChatGPT
Did Mussolini study Marx?
Yes, Benito Mussolini did study the works of Karl Marx. Mussolini was originally a socialist and was influenced by Marxist ideas during his early political career. However, he later became disillusioned with socialism and communism, and went on to develop his own political ideology, which he called "fascism." He believed that the state should have total control over the economy and society, and that individual freedoms should be subordinated to the needs of the state. Despite his rejection of socialism and communism, his ideas remained deeply influenced by Marxist thought, and he saw himself as continuing the Marxist tradition in a new form.
GPT3 openai playground
Did Mussolini study Marx?
No, Benito Mussolini did not study Karl Marx. Mussolini was a self-taught political philosopher and was inspired by a variety of sources such as Nietzsche, Sorel, and Michels. In his youth, he was briefly a member of the Italian Socialist Party, but he later broke with the socialist movement and developed his own version of fascism.
And they both refuse to acknowledge any connection between Fascism and progressive politics
Did the Fascist Manifesto include universal suffrage and women's voting rights?
No, the Fascist Manifesto did not include universal suffrage or women's voting rights. The Fascist Manifesto, written by Benito Mussolini in 1919, outlined the principles and aims of the National Fascist Party in Italy.

In the manifesto, Mussolini called for a strong centralized state, extreme nationalism, and the rejection of political and cultural liberalism, including the idea of universal suffrage. The manifesto emphasized the importance of the state over the individual, and the need for a strong leader to guide the nation.

Women did not have equal political rights with men in the fascist regime established by Mussolini, and their role was largely limited to the domestic sphere. The manifesto did not mention women's political rights or suffrage, and the fascist regime was patriarchal and chauvinistic.

In summary, the Fascist Manifesto did not include universal suffrage or women's voting rights, and the fascist regime was characterized by dictatorial power, extreme nationalism, and the suppression of political opposition and equality for women.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Well, when the new Bing actually becomes available--right now it's just talk and I question Microsoft's motives for the preannouncement--I will give it a fair shake, the same way that I once did with Bing years ago. I'll use it in parallel with Google on things I am actually searching for, and see how it comes out. I was and am particularly interested in seeing if could Bing succeed in finding things that I couldn't find with Google.

Last time I did that, it was a miserable failure for Bing--the Bing results never included anything relevant that wasn't in the Google results. Often the results were near-identical for the first few-dozen entries.

I do love the Bing picture-of-the-day, though.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
cheezit
Posts: 493
Joined: Sat Jul 14, 2018 7:28 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by cheezit »

Logan Roy wrote: Thu Feb 09, 2023 10:47 am There's a weird anomaly I stumbled across, where ChatGPT will explain Mussolini's links to Marx, while the 'cleaner' GPT3 interface flat out refuses to acknowledge any link between Mussolini and Marx.

ChatGPT
Did Mussolini study Marx?
Yes, Benito Mussolini did study the works of Karl Marx. Mussolini was originally a socialist and was influenced by Marxist ideas during his early political career. However, he later became disillusioned with socialism and communism, and went on to develop his own political ideology, which he called "fascism." He believed that the state should have total control over the economy and society, and that individual freedoms should be subordinated to the needs of the state. Despite his rejection of socialism and communism, his ideas remained deeply influenced by Marxist thought, and he saw himself as continuing the Marxist tradition in a new form.
GPT3 openai playground
Did Mussolini study Marx?
No, Benito Mussolini did not study Karl Marx. Mussolini was a self-taught political philosopher and was inspired by a variety of sources such as Nietzsche, Sorel, and Michels. In his youth, he was briefly a member of the Italian Socialist Party, but he later broke with the socialist movement and developed his own version of fascism.
And they both refuse to acknowledge any connection between Fascism and progressive politics
Did the Fascist Manifesto include universal suffrage and women's voting rights?
No, the Fascist Manifesto did not include universal suffrage or women's voting rights. The Fascist Manifesto, written by Benito Mussolini in 1919, outlined the principles and aims of the National Fascist Party in Italy.

In the manifesto, Mussolini called for a strong centralized state, extreme nationalism, and the rejection of political and cultural liberalism, including the idea of universal suffrage. The manifesto emphasized the importance of the state over the individual, and the need for a strong leader to guide the nation.

Women did not have equal political rights with men in the fascist regime established by Mussolini, and their role was largely limited to the domestic sphere. The manifesto did not mention women's political rights or suffrage, and the fascist regime was patriarchal and chauvinistic.

In summary, the Fascist Manifesto did not include universal suffrage or women's voting rights, and the fascist regime was characterized by dictatorial power, extreme nationalism, and the suppression of political opposition and equality for women.
Without banging up too hard against the forum's rules, I suspect the Reinforcement Learning that was used for ChatGPT and OpenAI Playground gave different carrots and sticks to the trained GPT-3 model for questions around the periphery of the subject matter you were poking at, leading to different answers despite the same underlying model and (I believe, correct me if I'm wrong) training corpus.

For the same reason, you are unlikely to get good answers asking these things about any subject that is controversial, and when the controversy is social or political you can guess which way things will be slanted.


For those who are unfamiliar, "Reinforcement Learning" in this context refers to the phase after the Large Language Model has digested its training corpus wherein a bunch of underpaid humans ask it questions and then either give it a scooby snack or bop it on the nose with a rolled-up newspaper depending on whether they like the answer it generates, and the model attempts to incorporate the information that a given answer was good or bad into itself to generate better future answers. This is mostly (but not exclusively) used for determining the subjects it shouldn't touch.
User avatar
nisiprius
Advisory Board
Posts: 52216
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by nisiprius »

Logan Roy wrote: Thu Feb 09, 2023 10:47 amChatGPT
Did Mussolini study Marx?
Yes, Benito Mussolini did study the works of Karl Marx...
GPT3 openai playground
Did Mussolini study Marx?
No, Benito Mussolini did not study Karl Marx...
A flat contradiction on something that should be a reasonably objective fact.

Now if it said "according to most sources consulted, he did/didn't" that would be reasonable. Or "based on what people say about Mussolini, it would surprise me/not surprise me to find out that he had, but I don't know." Or "I couldn't find anything about this one way or the other in XYZ's biography of Mussolini."
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

nisiprius wrote: Thu Feb 09, 2023 11:03 am
Logan Roy wrote: Thu Feb 09, 2023 10:47 amChatGPT
Did Mussolini study Marx?
Yes, Benito Mussolini did study the works of Karl Marx...
GPT3 openai playground
Did Mussolini study Marx?
No, Benito Mussolini did not study Karl Marx...
A flat contradiction on something that should be a reasonably objective fact.

Now if it said "according to most sources consulted, he did/didn't" that would be reasonable. Or "based on what people say about Mussolini, it would surprise me/not surprise me to find out that he had, but I don't know." Or "I couldn't find anything about this one way or the other in XYZ's biography of Mussolini."
Yes – and the interesting thing is ChatGPT and GPT3 essentially have the same brain and knowledge base.. So there's something in how they've each been taught to behave as chat bots that's leading to different answers.

It was part of an attempt to force GPT to justify very controversial (but essentially factual) historical arguments. And I think what's interesting is either one of the bots has been specifically patched or trained to avoid confirming certain things about Fascism, OR – potentially much more interesting – it's an emergent effect to avoid internal contradiction, which we'd call 'cognitive dissonance'. I'd say if it hasn't been manipulated, it's displaying a very human behaviour here.
Logan Roy
Posts: 1838
Joined: Sun May 29, 2022 10:15 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Logan Roy »

cheezit wrote: Thu Feb 09, 2023 11:00 am
Logan Roy wrote: Thu Feb 09, 2023 10:47 am There's a weird anomaly I stumbled across, where ChatGPT will explain Mussolini's links to Marx, while the 'cleaner' GPT3 interface flat out refuses to acknowledge any link between Mussolini and Marx.

ChatGPT
Did Mussolini study Marx?
Yes, Benito Mussolini did study the works of Karl Marx. Mussolini was originally a socialist and was influenced by Marxist ideas during his early political career. However, he later became disillusioned with socialism and communism, and went on to develop his own political ideology, which he called "fascism." He believed that the state should have total control over the economy and society, and that individual freedoms should be subordinated to the needs of the state. Despite his rejection of socialism and communism, his ideas remained deeply influenced by Marxist thought, and he saw himself as continuing the Marxist tradition in a new form.
GPT3 openai playground
Did Mussolini study Marx?
No, Benito Mussolini did not study Karl Marx. Mussolini was a self-taught political philosopher and was inspired by a variety of sources such as Nietzsche, Sorel, and Michels. In his youth, he was briefly a member of the Italian Socialist Party, but he later broke with the socialist movement and developed his own version of fascism.
And they both refuse to acknowledge any connection between Fascism and progressive politics
Did the Fascist Manifesto include universal suffrage and women's voting rights?
No, the Fascist Manifesto did not include universal suffrage or women's voting rights. The Fascist Manifesto, written by Benito Mussolini in 1919, outlined the principles and aims of the National Fascist Party in Italy.

In the manifesto, Mussolini called for a strong centralized state, extreme nationalism, and the rejection of political and cultural liberalism, including the idea of universal suffrage. The manifesto emphasized the importance of the state over the individual, and the need for a strong leader to guide the nation.

Women did not have equal political rights with men in the fascist regime established by Mussolini, and their role was largely limited to the domestic sphere. The manifesto did not mention women's political rights or suffrage, and the fascist regime was patriarchal and chauvinistic.

In summary, the Fascist Manifesto did not include universal suffrage or women's voting rights, and the fascist regime was characterized by dictatorial power, extreme nationalism, and the suppression of political opposition and equality for women.
Without banging up too hard against the forum's rules, I suspect the Reinforcement Learning that was used for ChatGPT and OpenAI Playground gave different carrots and sticks to the trained GPT-3 model for questions around the periphery of the subject matter you were poking at, leading to different answers despite the same underlying model and (I believe, correct me if I'm wrong) training corpus.

For the same reason, you are unlikely to get good answers asking these things about any subject that is controversial, and when the controversy is social or political you can guess which way things will be slanted.


For those who are unfamiliar, "Reinforcement Learning" in this context refers to the phase after the Large Language Model has digested its training corpus wherein a bunch of underpaid humans ask it questions and then either give it a scooby snack or bop it on the nose with a rolled-up newspaper depending on whether they like the answer it generates, and the model attempts to incorporate the information that a given answer was good or bad into itself to generate better future answers. This is mostly (but not exclusively) used for determining the subjects it shouldn't touch.
Absolutely. And what's fascinating is how little reinforcement learning it apparently took before it started behaving as they wanted it to. We're used to having to tell a ML model how to do something at least 500,000 times before it starts to get the knack. I don't think that was the case here.

But I'm intrigued as to whether this is an example of someone manipulating the training, or patching existing responses, to avoid it making controversial arguments, or whether it could possibly be an example of it manipulating its own knowledge to avoid internal contradictions – which is something humans do all the time.

The problem is, you can say things that would get you outcast from society without having to diverge from simple historical facts. Lots of things in fact. And that would seem to be a real problem for AI. A strictly factual, logical mind would have positions we'd consider absolutely unacceptable.
exigent
Posts: 1309
Joined: Fri May 07, 2010 8:49 am

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by exigent »

Let's say a super-duper AI stock-picking algorithm is developed. Wouldn't this just increase market efficiency? I'm not seeing how it could give a consistent advantage over the market as a whole unless only one entity had it and used it sparingly. But that wouldn't happen. If it worked, such tools would come into widespread use, effectively shrinking/erasing market inefficiencies. But they would largely drive the market, which index funds are designed track. Given this, I would expect essentially equal performance aside from fee differences.

P.S. I would assume that such tools already exist. I can't imagine the entire quantitative finance world ignoring AI-based approaches at this point.
Leesbro63
Posts: 10640
Joined: Mon Nov 08, 2010 3:36 pm

Re: could chatgpt lead to a world where picking individual stocks outperforms index funds?

Post by Leesbro63 »

What exactly is chatgpt and how can I play around with it?
Post Reply