For those of you not following the Thief forums, there's now a proper AI audio replicator, and it's incredible. Upload a voice recording, and it will generate anything you want with that voice almost perfectly
Welp, here's the rendition of Principal Skinner reading a paragraph on action sentences from the book I'm reading that you didn't know you wanted to hear: https://voca.ro/15AYghHnUfvl
We're really in uncharted waters now, folks.
Edit: By the way, here's the link to the site running the tech so you don't have to dig for it: https://beta.elevenlabs.io/
Last edited by demagogue; 31st Jan 2023 at 21:53.
I believe these guys did that 6 years ago:
https://www.descript.com/lyrebird
More recently there was this:
https://www.theverge.com/2022/6/23/2...ad-relative-ai
I think it's creepy.
Last edited by heywood; 31st Jan 2023 at 19:37.
I knew of other programs years back, but they were all crappy. This is eons ahead of the ones I had seen
All the stuff for automated NPC conversations in RPGs is really coming together. People are already linking up GPT, voice to text and text to voice in Unreal Engine mods etc, so that you can actually converse with NPCs like real people.Originally Posted by Azaran
How that could work with the game's lore as far as I understand it, is about how they know which context-relevant data to send to the model. Basically, you break the lore into chunks (could be automated), then each chunk gets run through the language model, to generate an embedding (basically, a 1D vector which describes the context of the lore). You then store these chunks of lore along with their embeddings, and when you have a request, that request gets embedded too, then you just search the database for lore chunks which are close to that embedding and those get sent to the language model as additional data.
I'm pretty sure this is how they're doing it with ChatGPT any time it talks as if has encyclopedia type knowledge. The language model itself hasn't internalized all of that, they're kinda "cheating" by finding the appropriate "cheat notes" then uploading them along with your prompt.
So yeah, that would be great to use for NPCs to give them access to the background lore for your world in the game. Imagine some future version of Dwarf Fortress or something, with procedurally generated world history, but on top of that it has AI-driven chatbots for the NPCs which get fed the appropriate snippets of world lore automatically.
Yeah I remember dreaming of that back in the day, and now it's pretty straightforward to see the pipeline. What's significant is how high quality it can be, not to mention how quickly it's generated.
The only thing is that the models for ChatGPT and Prime Voice are probably in the teens to 10s of GB; so if you didn't want to store that locally, the game would have to call it remotely. But that's also a thing that's standard anymore.Then again, so is installing games with large file sizes.
But even with all that said, if it's for a focused game, you might be able to get away with pretty small models (just a few GB) just for what's relevant to the game, and it'll probably still work great for the purposes of the game.
I guess we can't forget that procedural art and gameplay can also be a part of it. I think soon enough this is gonna be a whole genre unto itself, where we have to learn new rules. While I can see a lot of it being sludge, I can also see a lot of creative people figuring out how to wrestle really good writing and content with it as well.
No, the scale is more than that. GPT-3 had 175 billion parameters. If you use 32 bit, then it would be 700GB of RAM required to hold a model. However they probably want to use 64 bit values, so that's 1.4 TB of RAM needed.
ChatGPT is probably bigger than that, but they've said GPT-4 will only be about 50% more parameters than GPT-3. Which makes sense, since 2 TB of RAM is basically the limit for high-end server motherboards. So they're effectively pushing up against the limits for regular server-grade hardware here, and that's what's constraining the models getting much bigger right now.
EDIT: You can find some articles (bullshit ones) floating around claiming that GPT-4 will be 170 trillion parameters, especially this one:
https://medium.com/geekculture/gpt-4...3-38c57f51e4e3
... Except that firstly, that's nonsense. And also, 170 trillion is 1000x the size of 175 billion, not 100x, so it's not even mathematically accurate nonsense.GPT-4–100X More Powerful than GPT-3
...
GPT-4 is significantly larger and more powerful than GPT-3, with 170 trillion parameters compared to GPT-3's 175 billion parameters.
Such a piece of software would require 1.36 Petabytes of RAM to store the parameters at 64-bit. This would mean the software could only be run on a handful of the most powerful supercomputers in the world, instead of being rolled out on commercial server hardware.
Here's the article about the top supercomputers in the world.
https://en.wikipedia.org/wiki/TOP500
It contains a top 10 list, and the 10th most powerful machine, Tianhe-2 in China only has 1,375 TiB total memory, so even it could barely fit a 64-bit 170 trillion-parameter model in RAM, and then only if you crammed both the GPU memory and CPU memory full of the parameters. So this is literally stuff that could only run on a number of billion-dollar machines worldwide you could count on your fingers.
Last edited by Cipheron; 1st Feb 2023 at 00:30.
Ah, I was thinking about the Stable Diffusion (AI art) model as a reasonable comparator, but I guess it makes sense it'd be a lot bigger for a language model. They call them large language models (LLMs) after all.
Okay, so short of some ridiculous memory storage tech in the near future (not ruling anything out anymore, but I'm not betting on it too soon), it'd need to be on the studio's servers and able to scale to a lot of users.
But I was already thinking that was going to become more standard in the near future anyway, cf. MSFS and some other big data games. Something like this would just accelerate that trend.
Last edited by demagogue; 1st Feb 2023 at 00:29.
Forget about training an AI to mimic a person's voice, that path leads into an ethical minefield. The missing piece is a parametric voice generator, where you can specify a gender, range, style, accent, and other parameters necessary to make a convincing but unique-sounding voice. At that point, voice acting as a profession is effectively dead.
Well guess what
In 2000, I played Deus Ex and enjoyed the in-game conversation with Morpheus. It was just a traditional dialogue tree, but it was deeper and more thoughtfully written than any NPC interaction I had in a game up to that point. Ever since then, I've been dreaming about getting rid of canned NPC barks and dialogue trees in RPGs and imm sims, and replacing them with a conversational AI back end.
Giving each character a unique and original voice is one of the challenges. Giving each character a unique and original personality is something entirely different.
ChatGPT can almost write a decent Monty Python sketch:
Title: "The Swift Watchmaker"
Scene: A watchmaker (Mr. Thompson) is working in his shop when the king (His Majesty) enters, accompanied by his court.
His Majesty: (excited) Mr. Thompson, I've heard that you are the most skilled watch maker in all the land. I need your assistance!
Mr. Thompson: (bowing) Of course, Your Majesty. How may I be of service?
His Majesty: (pulls out a pocket watch) I have a special request. I need you to make a watch that is so accurate, it will never lose a second.
Mr. Thompson: (impressed) That is quite a challenge, Your Majesty, but I am confident that I can create such a watch for you.
(Mr. Thompson sets to work, and several days later, presents the finished watch to His Majesty)
Mr. Thompson: (proudly) Here it is, Your Majesty. The most accurate watch in the world. It will never lose a second.
His Majesty: (excitedly) Let's see it in action!
(Mr. Thompson winds the watch and sets it on the table. It starts ticking at a frantic pace, so fast that it sounds like a buzz)
His Majesty: (stunned) What is this? This is not a watch, it's a blur!
Mr. Thompson: (realizing his mistake) Oh, my. I seem to have made it a bit too quick.
His Majesty: (disappointed) This is unacceptable, Mr. Thompson. I needed a watch that was accurate, not one that runs faster than a horse.
Mr. Thompson: (embarrassed) I am so sorry, Your Majesty. I will fix it, I promise.
His Majesty: (sighing) I hope so, Mr. Thompson. I have important events that require punctuality, and this watch is worthless to me.
(Mr. Thompson takes the watch back to his shop and sets to work. Several days later, he presents a corrected watch to His Majesty)
Mr. Thompson: (nervously) Here it is, Your Majesty. The corrected watch.
His Majesty: (cautiously) Let's see it in action.
(Mr. Thompson winds the watch and sets it on the table. It ticks at a slow, steady pace)
His Majesty: (relieved) Ah, much better. This is what I was looking for.
Mr. Thompson: (bowing) I am so glad I could meet your expectations, Your Majesty.
His Majesty: (smiling) Yes, you have done well, Mr. Thompson. And as a reward, I will give you a gold coin for every second this watch runs without fail.
Mr. Thompson: (delighted) Thank you, Your Majesty!
(The king pockets the watch and starts to walk out of the shop, but his steps become faster and faster with each tick of the watch. He tries to slow down, but his feet are moving out of control)
His Majesty: (panicking) Help! What have you done to me, Mr. Thompson?
End scene.
If I were young and entering the job market for the first time in the next 3 to 5 years or later, I would start putting significant time & effort into studying & practicing how to work with these different AIs to curate & optimize their output or response.
That skill alone may become more important than any individual skill being executed by an AI.
And this appears to be headed for every field. I anticipate Cipheron being spot on about a new kind of digital divide developing.
You know this would happen
I heard a related post, an AI recording (using Eleven AI) of David Attenborough spouting the most vile racist vitriol known to man. It's frighteningly convincing. I imagine this will have a huge impact on scandals, the justice system/audio evidence, etc. People actually caught on recordings saying things they shouldn't will claim it's AI generated, etc.
Oh yeah, we could see that coming.
I think we'll adjust to deep fakes rather quickly. Existing cryptographic technology like certificates and blockchains already provide what's necessary to answer questions of authenticity and provenance when it's important. People will choose to believe what they want despite it.
And Twist, my head is nodding. Machine learning and gene editing are the most transformational technologies I've seen since the WWW. And you can play with ML at home.
Last edited by heywood; 1st Feb 2023 at 17:46.
I guess this is flirting with further abuse, but I hope one of the features in the works is to take an existing vocal track and have it replaced with a custom voice. I'm thinking about the pipeline for making new characters in games, like for Dark Mod, but it'd apply to any game a person is making, or even if they want to mod new characters into other people's games.
If you've ever dug through the audio files of a game, there will typically be 100s or 1000s of audio snippets for each character that can easily take 10s of hours of recording. Usually when you want to make a new character, the really hard part isn't the modeling and animation so much as making a new complete audio set because of all the hours of recording, it has to be on the right equipment, if you mess up you have to have the person come in or do it themselves on another day, and you probably can't add to the set after the initial recordings, etc.
But if you could just batch process an existing set with a new character's voice, that would make that whole pipeline a whole lot easier. I guess you'd want a lot of new dialog, and you can still do that too. But as a quick job, it's probably easy to use an existing set, plus you can't really get the proper prosody (emotion and emphasis in voice) in with the text prompter, and there are a lot of things like grunts, breaths, and death screams where the change in voice is enough, and for a lot of things you can probably just shuffle phrases around and it'd be novel enough.
-----
Edit: And if you have that much, the pipeline is also easy to see how you could replace people in TV shows and games with a few photos of them and a 2 minute clip of them talking. I somehow feel like this is going to become really common really soon.
Last edited by demagogue; 1st Feb 2023 at 22:24.
This is gold, Larry! Gold!!
https://www.twitch.tv/watchmeforever
This comment from the 4chan thread on the audio replicator says it all
The real answer is they knew exactly what was going to happen, and counted on it. Unfiltered launch would drive the numbers through the roof, which they can then show their investors to gouge more money out of them, while reaping the bonus brownie points for tightening the screws.
I was thinking along similar lines demagogue. Text to speech has come a long way, but can you give them acting prompts? Can you ask them to say something in a way that expresses emotion?
I uploaded Krusty the Clown's voice, and the voice set that came back completely cut the affectation out of his voice, so it was just an emotionless, flat voice, if you can believe it. It's like you're just listening to the voice actor's normal voice.
That said, if you drop the stability and similarity sliders, use expressions that typically have emotional valence, and repeat lines enough, you'll get some tracks with some emotion. And I think the way it works is that once it's got a valence, it sticks with that for the rest of the track. So if you can get it really worked up for the first line, you can get the emotion into later lines. (It also works the other way around where, if the first line is low energy, the whole track will be.)
It'd be better if there were prosody sliders; but as they charge by the word, it's kind of easy to see why they went with a system where you have to make a track 8 times until you get the voice as close as you can get to what you want.
Oh, you know what will enjoy a huge revival because of this? Prank calls. You'll now be able to clone Joe Pesci or Arnold's voice for a whole new era of epic prank calls
AI generated video is taking off too.
This one is available, but very wonky
https://replicate.com/nateraw/stable-diffusion-videos
I'm sure procedural animation from scratch may get there at some point, but in the meantime I think it's better to start with a parameterized base, like models and an environment in an Unreal game that can be initialized by AI, and then AI dynamically animates them from there, all within some reasonable or realistic ranges (like e.g. what Endorphin used to do for NPC animations, but now ML built).
After looking at that perpetual Seinfeld episode above, which any time I've gone to it always have over 10,000 viewers at any given time, I have an idea to make my own perpetual show. I think the pipeline for it wouldn't be that hard with what we already have.
I found it fascinating at first, but after 2 minutes just feels like watching paint dry. Hopefully it's an evolving thing that can improve over time, and come up with better jokes
People need to build up their prompting skills if they're going to do this with the current state of the tech.
Posting Permissions
Reply With Quote