Inpainting is a technique where you start with a picture (in this case me) and ask the AI to replace a specific part of it. My first test was to try to add a second nose to myself, so I could smell things better. (All pictures/modifications created with OpenAI’s DALL·E 2 – all “original” pics used for inpainting are owned by me)
“Man with second nose growing out of forehead”
Um, not what I was going for.
Let’s stay away from the face and try to change Akiko’s dress. I use the brush to remove the area below her neck, slightly larger than the area of her dress.
“A beautiful elegant blue dress”
Incredible. It also filled in small parts of the person/stroller that is out of focus behind her as needed.
Retro actively changing clothes in any picture? Useful.
Another test. We’ll replace my son’s baseball glove with a pizza.
These results aren’t as convincing (I should have given it more arm to replace for a more natural pose I think), but I do appreciate how it was able to add missing detail to his hair after the glove was removed.
Well, you know what I have to do…
“Extremely muscled handsome man without a shirt”
Hmm. Not great. I guess changing my body into one with muscles is just one step too far for today’s technology. My guess is Dall-e actually doesn’t know much about bare skin (nipples and belly button seem to be missing?) due to using a censored training set to stop it from.. well, doing certain things. I’ll bet a suit works better, let’s try that.
“Three people wearing classy colorful suits.”
This time I did all three of us is one go. Not perfect – I wonder if it would work better if I did it in three separate operations, one for each person? Hrm. If you’re curious how I set which areas it could fill, it looked like this:
I left the holding hands in as a constraint for the image created.
Random Dall-e 2 test pictures
More random pics with the prompts used, some prompts created by friends.
“pyramid head on late night with conan o’brien”Another try: “Still from silent hill movie, pyramid head on late night with conan o’brien, dramatic lighting”“flaming headed monkey holding paperwork”“hot dog being thrown down a hallway”“winnie the pooh and Confucius in the delorean from back to the future”“Winnie the pooh and Confucius in the DeLorean, a still from the movie back to the future, 4k”“Winnie the pooh fixing the flux capacitor, still from the movie Back To the Future”
I’ve been interested in AI and computer generated media forever. As a kid, I created a stupidly naïve “AI” on my Commodore 64. I just programmed text responses to hundreds of inputs. “Hello” gave back “Hello, how are you.”.“I’m fine” gave back, “Great, me too.” and so on. I proudly showed my parents how smart my computer was.
A hollow illusion then, reality today. From Eliza to Deep Blue, Watson to Siri, between Deepfakes and GPT-3 (and friends), it’s all coming together to change our world at a blinding pace.
Deepfake test I did on my local computer a few years ago with DeepFaceLab
Do you know what two computer things I’ve been playing? Eldin Ring and OpenAI’s GPT-3 Playground. I’ve spent about $70 on each last month. Does spending cash on AI seem weird?
It’s no exaggeration to say playing around with it is one of the most engrossing and creative experiences I’ve had with computers to date.
OpenAI Inc/OpenAI LP are the big dogs in the field and carefully police usage. You can probably think of dozens of ways this technology (both the text creation and text to image tech of DALL-E) could be used for evil, spam, and misinformation.
Recently I’d been playing around with DALL·E mini and was in the process of setting up my own local server to try to allow higher quality work when I was granted access to the holy grail: DALL·E 2.
Let’s have the AI generate images for my old text-only game
In 1989 I started work on the BBS door-game Legend Of The Red Dragon. It’s a text game. What would happen if I took text from that game and asked AI to draw pictures for it from only the game text?
Let’s try its opening text as a prompt:
“You wake up early, strap your Short Sword to your back, and head out to the Town Square, seeking adventure, fame, and honor.”
Huh. Looks like it could use more direction. Let’s add “In the style of 3d rendered adventure game.”
“You wake up early, strap your Short Sword to your back, and head out to the Town Square, seeking adventure, fame, and honor. In the style of 3d rendered adventure game.”
Not bad. How about the Red Dragon Inn? Wonder how long these text prompts can be, let’s try the whole thing.
“You enter the inn and are immediately hailed by several of the patrons. You respond with a wave and scan the room. The room is filled with smoke from the torches that line the walls. Oaken tables and chairs are scattered across the room. You smile as the well-rounded Violet brushed by you…”
Well, the raw weird prose doesn’t seem to work that well. It isn’t given enough information to know it isn’t modern day. What if I change it around a little bit… (in theory you could use AI to rewrite the sentence to not be 1st person and add keywords to help with theme and historic era)
Note: I blocked out a woman’s face, I thought the rule was we can’t show them – but maybe we can, need to check the dalle-2 rules again.
“A painting of the medieval Red Dragon Inn. The room is filled with smoke from the torches that line the walls. Oaken tables and chairs are scattered across the room. Violet the barmaid smiles.”
Let’s try a different visual style.
“A photo of the medieval Red Dragon Inn. The room is filled with smoke from the torches that line the walls. Oaken tables and chairs are scattered across the room. Violet the barmaid is working.”
Hmm, it’s obvious that I could get better results if I took more care in the prompt text, but nifty anyway.
I could see it being fun to play old text games with AI generated images. I don’t see how to control Dall-e 2 with an API at the moment otherwise I might try modifying an infocom interpreter to automatically fetch them during play.
The 10-20 seconds to generate an image wouldn’t be fun to do it live, but how cool would it be to see “a bucket is here” and it appears/disappears from the image as you play the game?
The big problem is uniformity of style – but there are some tools dealing with this I haven’t played with yet. (starting with an uploaded photo, for example)
Let’s use AI for everything
How about using AI to help generate a brand new story, then illustrating it too?
Here is a test. The text with the white background I typed. The text with the green background was generated by AI. (Specifically, OpenAI.com’s text-davinci-002 engine)
Ok, we now have two characters. Now, we keep this text and continue with more prompts, interactively pulling out more details. We can always undo and try different prompts if needed.
Ok, now let’s send these descriptions to DALL·E 2 to create AI-based visual representations of the story the AI created. First let’s do Feival’s house:
“Feivel’s house is small but cozy. It is made of sticks and stones, with a thatched roof. There is a small fireplace in one corner, and a bed in another. A few shelves hold some of Feivel’s belongings, including his treasured map of the area.”
Not bad. I like the sixth image because I can see the treasure map on the chair.
Let’s do the Thimble description next. This time I’ll add “Realistic photo” at the end to specify the kind of image we want.
Thimble is an elderly mouse with gray fur. She is small and frail, but her eyes are bright and full of wisdom. She wears a simple dress and a scarf around her neck. She walks with a cane, but despite her age, she is still quite spry. Realistic photo.
Hmm. The cane didn’t seem to quite make it. This story seems like it might make a good children’s book. Let’s add “by Richard Scarry” to get that type of art style.
“Thimble is an elderly mouse with gray fur. She is small and frail, but her eyes are bright and full of wisdom. She wears a simple dress and a scarf around her neck. She walks with a cane, but despite her age, she is still quite spry. By Richard Scarry.”
Definitely got a children’s book style! The cane is now in every picture. I like this style.
I can ask for more variations in this style:
Writing a story with the characters the AI created
Hmm. Ok, we’ve got our stars, let’s have the AI write a story using them. I’m adding “Write an amusing children’s book about the above characters with a twist ending. Chapter 1:” to the end of the text we’ve already generated. (Again, green parts were created by the AI)
Well, uh, it’s a story. There are ways to coax out longer and more interesting things but this is fine for this test. Just for fun, let’s see if we can create artwork for the amazing battle scene of the giant mouse trap catching cats. I’m going to cheat and use my own descriptions for the prompt.
“Evil cats that wear clothes being cause in a giant mouse trail as a tiny clothed hero mouse strikes a victory pose in detailed colored pencil”
Uh, ok, obviously that prompt isn’t great as it looks like a cat is being hit with colored pencils. I’m showing you my failures, not just the good ones here! Let’s forget the mouse and just focus on the cats and the mouse trap.
“Evil cats being caught in a giant mousetrap, in surrealistic art style.”
These are pretty wild! Some of the faces are .. I don’t know, it may have actually tried to drawn them injured by a mousetrap, in retrospect this could trigger unintentionally gory results, especially if I used ‘photorealistic’ as a keyword.
Let’s move to safer ground and create an image for the happy (?) ending.
“An old clothed grandma mouse with a cane holding hands with a brave little boy mouse . Art by Richard Scarry”The end!
Random fun with DALL·E 2
These are just various pictures created with DALL·E 2 and the text prompts used. It’s very interesting to see AI interpretations. Special thanks to Kevin Bates for brainstorming these prompts with me. It’s addicting, I can’t stop!
Note: The six images pic shows the prompt used, then I do some “closeups” of the more interesting ones. It’s really fast to do it this way, sorry it’s not nicer so each little pic is clickable.
“portable open source video game system”Not real sure about this D-PAD designDon’t steal these designs, Nintendo“the droids from the movie starwars”“R2D2 with arms and legs giving a high-five, zoomed out, photo”“Ewok from the movie Return of the Jedi in a bikini”“surrealistic photo of a puppy waring a VR helmet in a futuristic spaceship”“the abstract concept of having free will”“fisher price guillotine”“golden gate bridge in the style of an oriental scroll”
In Summary..
Well, I’ve put way too many pictures in this post so I’ll end it here. The AI models I used are top of the line and have many usage restrictions, but it’s only a matter of time before similar things are available to everyone – Good or evil, unrestricted. I’m simultaneously excited and worried.
If you want to play around with generating images yourself, try DALL·E mini. Its output isn’t as impressive but it’s still fun and interesting to play with.
Ok, this is one of those posts that are more just to document something so the next time I install VS on a computer I remember how to do this and can find the file.
If you don’t use Visual Studio C++ then you should run. Or, prepare to dive into the incredibly boring world of the lengths programmers will go to just to avoid monotonous typing.
I’m all about classes. When you think of classy, think of me. Any time I do anything, I need a .cpp and.h file for it. And I want ’em setup pretty with the current date and stuff. Automagically!
See that? If I specify “GameProfileManager”, these two files get made with these contents. That’s all I’m trying to do, as I make a LOT OF CLASSES.
For years I used custom macros I’d made for Visual Assist, but at $279 + $119 yearly it’s a bit hard to continue to justify using VA when VS 2022 has most of its features built in these days.
Yes, you can pay less but rules like the payment “Cannot be reimbursed by a company or organization” and the world’s most annoying licensing system makes it not worth it.
So what’s a lad to do? Well, I finally got off my bum (well, on it, I guess) and learned how to have the same macros with built-in features that VS has had forever. They work a tiny bit different but close enough.
Hint: Don’t even try to use the “Snippets” feature to do this, it won’t work.
How to add
Downloadthis file and place it in your %USERPROFILE%\Documents\Visual Studio 2022\Templates dir.
(or %USERPROFILE%\Documents\Visual Studio 2019\Templates if you’re using that)
DON’T UNZIP! VS reads it as a zip.
Close and restart Visual Studio.
Then add new classes using the “Add item” menu. Like this shittily produced unlisted video shows: (can you guess the music I used? Why did this even need music? Are you wondering what youtube video I was watching?)
How to change it
Great! Ok, but you probably wanted to customize it with your own name and not give me credit to everything you make. Well, fine, if you must.
To modify it with your own name/email/texts, just unzip the file you downloaded, you’ll see four files inside. Delete the original .zip. (you’ll make your own in a sec)
Edit the .cpp and .h as you like, then zip the four files up again. (I don’t think the name matters, it just has to be in the same directory as before) Restart VS and you should see the changes.
When editing, there are many special words you can use that get replaced automatically (full list here).
Some useful ones:
$safeitemname$ – (current filename without the .cpp or .h part)
$projectname$ – The name of the project this is in
$year$ – Current year (ie, 2022)
$time$ – The current time in the format DD/MM/YYYY 00:00:00
$username$ – The current user name
$registeredorganization$ – The registry key value from HKLM\Software\Microsoft\Windows NT\CurrentVersion\RegisteredOrganization
Hint:Edit the MyTemplate.vstemplate file to change the name and some other things. Useful if you want multiple templates, one for component classes, etc.
And that’s it! Suck it, Visual Assist. But also I love you so please get cheaper and less annoying to buy & license.
The PlayStep Mini is a tiny device that allows a parent to limit screen time from any HDMI device based on “charging up” energy using an exercise stepper.
According to the CDC, kids aged 8 to 10 spend an average of 6 hours a day in front of screens.
While rationing time is good, in the real world sometimes parents need additional tools to help keep kids fit.
In 2019, I made the PlayStep, an open source program that runs on a Raspberry Pi that allows a kid to power the screen with exercise. Years later, we still use it for one of my kids!
But I guess I’m the only person on earth using it, even though it’s open source. I don’t know why, I mean it only costs around $700 in hardware and low level knowledge of linux command line stuff…
Oh, you want something more reasonably priced and easier to use?
Behold: The Playstep Mini!
The differences are this:
No cool video mixing, just on and off
Zero latency
Extremely power efficient (uses 30ma directly from the input HDMI’s 5v power)
Standalone and easy to use
Can be made for under $10 (plus exercise stepper that you might already have)
If this is something you’d be interested in buying as a kit to use, well, leave a comment, I can buzz you if myself or someone else ever offers it for sale.
If you’re someone who knows if it’s legal to sell something like this as a kit without certification of any kind, uh, also please let me know.
F.A.Q.
Won’t a kid just disconnect it and play without exercising?
If mine did, he knows he’d probably lose gaming altogether for a week at least so it’s never been an issue here.
Is it safe? Will my kid get a leg injury from too much exercise?
Please use common sense and adjust the ‘time energy’ per step, step resistance on the machine, and “max energy level” to something that is safe. Uh, if your kid is limping around you should definitely disconnect it.
Consult your doctor or whatever so I don’t get sued when your kid is skinny but with two herculean calves.
Can I get some made from JLCPCB and give them away or sell them?
Sure, I don’t care, do anything the GPL license allows. I could switch it if there is a reason.
You freaking used pcb auto-routing with HDMI? My Eyes! My God man, I would love to help you do it right
Hey, I’m a beginner! But that is very nice of you to offer to help improve it, you glorious human.
This version works great for 1080p, but the board layout is too stupid and noisy for 4k to work. I guess I should stop using auto routing and get rid of all the HDMI vias by using more layers or whatever?
If you can help me work on this and you’re ok with its open and free hardware license, please check out the github project.
A Looking Glass holographic display device connected to a Windows computer via hdmi (designed for the Portrait, but in theory it should run on all of them…)
The Holoplay driver installed
A beefy ass graphics card
A game rom (.a26 or .nes file) which needs to be put in the /atari2600 dir or /nes dir, depending on the system it’s for
If you don’t have a holographic device the screen will look like blurry garbage. If you want a build that works in 2D for some reason, I guess I could do one though…
Supported games (you must supply the rom, they are not included in the zip):
Note: Supported means they are playable, it doesn’t mean they work perfect or don’t have issues…!
Pitfall! (Atari 2600)
Super Mario Bros (NES)
Castlevania (NES)
Jack Bros (VB)
Panic Bomber (VB)
Teleroboxer (VB, lickers between rounds)
Vertical Force (VB)
Wario Land (VB)
The actual filenames don’t matter, it detects supported games by checksum.
Note: There are two versions of Castlevania NES roms out there, if it’s not detected and not playing in 3D you might have the wrong one.
It will run unsupported games, but they won’t have any depth/3d effects.
I’ve only tested with the Looking Glass Portrait and the OG 8.9. They both seem to run around the same speed, with the OG 8.9 being too zoomed in initially. (try the – key to zoom out?)
When it starts, it will show a menu with the hotkeys, but here they:
Arrow keys: Move Ctrl, Space, Enter: A and B and Start buttons (gamepad supported too) Return: Reset game Num 0 through 5: Set frameskip (higher makes the game run faster by not showing every frame) Num 6 through 8: Toggle texture smoothing, shadows, lighting P: Pause A: Adjust audio to match game speed (experimental but can help with audio problems) –/+: Zoom in/out. Hoping this will help with other Looking Glass sizes. S: Save state L: Load state < and > : Cycle through detected games (any roms you’ve placed in the atari2600 and nes directories)
Tips
I can get 60 fps with everything with a 5ghz with Nvidia 3090. Exception is Virtual Boy which I’ve limited to 50fps to match the real device.
If a game is too slow, press 1 or 2 for frame skipping modes. If audio is weird, press A to cause audio to sync with the recent framerate. (gets rid of pops and scratches usually, but pitch/speed will be wrong making it… interesting…)
If you have problems, check the log.txt file for clues. (created in the root dir where HoloVCS.exe is)
FAQ
Q. Does it support other games besides these?
A. It will play unsupported games without 3d plane effects, so not really
Q. I noticed you’re using emulators via a libretro dll interface, does this mean I can pop in more emulator types?
A. Yes! Well, no. I mean, each requires customizations to work properly and do 3d stuff.
Q. The snake in Pitfall! is mostly invisible!
A. This is a known bug, sorry. I mean, it’s a ghost snake now
Q. Why do some levels look weird or broken?
A. Sorry, I only made it so the first levels works, didn’t worry about later stuff. It is possible to detect current level/environments via PPU memory and adjust rendering to match the situation though.
Q. Why is it called HoloVCS?
A. It originally only supported Atari VCS emulation. Too lazy to change it
History
1.0 Initial release, supports Pitfall! for the Atari VCS
1.1 Added support for a few NES games too
1.2 Added some Virtual Boy support. The neat thing about this is there nearly no game specific code happening, it’s reading the 3d position directly by hooking into the emulator at a low level. It’s not able to capture everything but it’s enough to be a neat gimmick. The only game-specific thing I’m doing with VB is setting the zoom level to look slightly better for certain games.