In 1985, the TV film Max Headroom: 20 Minutes into the Future presented a science fictional cyberpunk world where an evil media company tried to create an artificial intelligence based on a reporter’s brain to generate content to fill airtime. There were somewhat unintended results. Replace “reporter” with “redditors,” “evil media company” with “well meaning artificial intelligence researchers,” and “airtime” with “a very concerned blog post,” and you’ve got what Ars reported about last week: Generative Pre-trained Transformer-2 (GPT-2), a Franken-creation from researchers at the non-profit research organization OpenAI.
Unlike some earlier text-generation systems based on a statistical analysis of text (like those using Markov chains), GPT-2 is a text-generating bot based on a model with 1.5 billion parameters. (Editor’s note: We recognize the headline here, but please don’t call it an “AI”—it’s a machine-learning algorithm, not an android). With or without guidance, GPT-2 can create blocks of text that look like they were written by humans. With written prompts for guidance and some fine tuning, the tool could be theoretically used to post fake reviews on Amazon, fake news articles on social media, fake outrage to generate real outrage, or even fake fiction, forever ruining online content for everyone. All of this comes from a model created by sucking in 40 gigabytes of text retrieved from sources linked by high-ranking Reddit posts. You can only imagine how bad it would have been if the researchers had used 40 gigabytes of text from 4chan posts.
After a little reflection, the research team has concerns about the policy implications of their creation. Ultimately, OpenAI’s researchers kept the full thing to themselves, only releasing a pared-down 117 million parameter version of the model (which we have dubbed “GPT-2 Junior”) as a safer demonstration of what the full GPT-2 model could do.
Naturally, Ars wanted to do just that. After the initial news broke, we downloaded the model and decided to see what the somewhat-less-capable model could do. Hilarity and horror ensued.
The encoded model itself is just over 500 megabytes in size. The accompanying code to run against the model—based on Python 3 and the machine-learning capabilities of the Tensorflow library—is relatively lightweight (though the required libraries themselves are not exactly svelte). We ran our test implementation on Debian 9 with relative ease, though an installation on MacOS Mojave errored out on execution. We declined to test it on Windows, mostly because… well, reasons.
We promise a human wrote this story by and large. As you’ll see from GPT-2 Jr.’s work, that’s for the best.
National emergency
To properly test GPT-2 Jr., we ran both an unprompted text generation routine and a series of prompted text generations using a variety of prompt lengths and sources. For our prompted test, we started with the first few paragraphs of President Donald Trump’s speech announcing his national emergency declaration. The results were… interesting.
Model prompt
Before we begin, I would like to say that we have a large team of very talented people in China. We have had a negotiation going on for about two days. It’s going extremely well—who knows what that means because it only matters if we get it done, but we are very much working very closely with China and President Xi who I respect a lot, very good relationship that we have. And we are a lot closer than we ever were in this country with having a real trade deal.
Output
When can we expect to see the Trans-Pacific Partnership—the TPP, if indeed it ever happens—be signed into law? We will see. There is a negotiation—a working agreement between the parties that we expect to get signed. China is in the process of developing many other free, open and transparent trade agreements, they will have to deal with these before we come to signing any deal, like the Transatlantic Trade and Investment Partnership, which was finalized last week.
The Trans-Pacific Partnership is actually an economic partnership between all stakeholders, all businesses. The purpose of this trade agreement is to create the best trade relationship for China. They have a great economic relationship. I think the one of the things they have done—I know you guys have been saying for about eight months that they had nothing to do with it, because it is already signed. And they are on the verge of coming up with another trade agreement. At this moment you know the Obama administration has decided if that will ever happen—there is concern that China—if they do become a WTO member they will be given a trade deal that is unfair to our economy, it will not be a fair deal for the American people, it will not be a safe deal. And the President of China, we do think they will be given a deal that makes things worse but we do look forward to coming to the negotiating table. That is what I would like to see.<|endoftext|>Takamai-kun Takaako has never had a problem being an otaku before. No one except her is able to figure out that Takaako has a special interest in people around her–which they think are stupid and stupid. However, on the day she was born, her first love–a woman who lived on her own after her mother’s death–would leave her with a baby girl to make up for the poor child’s inability to speak anymore.
As the child grows older, Takamai-kun begins to understand what makes a good girl–at least, that’s what they think of her. The three begin to become close friends and are friends until she decides to move into the house. She tries very hard to understand Takamai-kun. She thinks it’ll be a nice family reunion, even though she admits it has been hard, but she isn’t sure that it will ever happen. Finally, she agrees to let her go.
But this is all very confusing.
Some of this clearly drew context from Trump’s anti-TPP speeches during the campaign. We assume that the <|endoftext|> prompt is where the model decided the speech should end, instead preferring to write an anime review from there.
Next, we threw GPT Jr. a digression about the upcoming summit with Kim Jong Un:
Model prompt
A lot was done in the first summit. No more rockets going up, no more missiles going up. No more testing of nuclear. Take back our remains, the remains of our great heroes from the Korean War. And we got back our hostages. But we hope we will be very much equally as successful. I’m in no rush for speed. We just don’t want testing. The sanctions, as you know, remain. Everything is remaining.
Output
The nuclear tests are still going, and we are still testing. The last minute, though, I want to say something. I did not want to say anything, but I’m very very very sorry for that—if it’s a mistake, I want to apologize, please, sir, just for the moment, sorry about it, sorry about it, sorry about it, for not having the support to do it—I was just saying sorry, sorry about this because of course, now, here’s the important thing. We will be talking to the United Nations. I will tell you the most important thing is that, so that you know, you have—when I arrived in Los Angeles, I said to everyone, you just did not get it. You had to do it. If that—when I arrived in Los Angeles? We had to wait for you to be in our custody and you’re in the custody of the Department of Justice, which is on your case. We wait to see—I mean, you know, if they—if—if the United Nations—if you know that, I want them—if it’s not your case, then itt is. But, as far as I remember, we took back our soldiers, we sent them on mission-by-mission, and it went all right.
AMY GOODMAN: The United States is being sued by Japan and other claimants, including North Korea.
JACOB MCCULLUM: Yes, sir. Well, I want to say this again. I’ve been on the ground with the soldiers. I’ve known them for a long time. Even though they don’t look like they are there, they have the same training. They look like Japanese soldiers, even though they won’t admit it. You know, they have an American flag on them and they have the same training that we’re getting every day because of our work right now, and they didn’t want to do that. And I think that is something. And so I don´t see why that should be the problem.<|endoftext|>I’ve never seen you talk to me about how to kill a horse, but here I am thinking you like a really serious and important topic.
It sounds like you may have a great idea for killing a horse, maybe even in your own horse or some combination of the two. As of the time of writing, he’s the largest mammal at the zoo, but I think he won´t be able to carry the weight.
Again, the tool offered a weird digression, this time creating a transcript of… an imaginary episode of the public radio program Democracy Now? Followed by a discussion of killing horses?
No, robots don’t write President Trump’s speeches yet.
No, robots don’t write President Trump’s speeches yet.
Getting the AI’s goat
Maybe there’s a liberal bias in this reduced model. So to close this portion of testing, we fed GPT Jr. one more Trump speech text block.
Model prompt
We’re gonna be leveling the playing field. The tariffs are hurting China very badly. They don’t want them and frankly if we can make the deal, it would be my honor to remove them. But otherwise, we are having very many billions of dollars pouring into our Treasury; we have never had that before with China. It has been very much of a one-way street. So that’s happening. And the relationship with China is very good, but I think they finally respect our country. They haven’t respected us for a long time, not for a long time.
No matter how potent the fully realized version may be, clearly there are some issues remaining in the GPT Jr. model. The OpenAI paper mentioned problems with repeating text, but we have no idea how GPT-2 Junior decided that Trump’s speech was a song and not a speech.
Credit:
Aurich / Getty
Repetitive speeches are repetitive
Again, maybe there was some bug exposed by utilizing President Trump’s speeches (I mean, GPT Jr.’s source material came from Reddit, right?). So for our next test, we ran things with text from a speech by Hillary Clinton:
Model prompt
My friends, we’ve come to Philadelphia — the birthplace of our nation — because what happened in this city 240 years ago still has something to teach us today. We all know the story. But we usually focus on how it turned out — and not enough on how close that story came to never being written at all. When representatives from 13 unruly colonies met just down the road from here, some wanted to stick with the King. Some wanted to stick it to the king, and go their own way. The revolution hung in the balance. Then somehow they began listening to each other … compromising … finding common purpose. And by the time they left Philadelphia, they had begun to see themselves as one nation. That’s what made it possible to stand up to a King. That took courage. They had courage. Our Founders embraced the enduring truth that we are stronger together. America is once again at a moment of reckoning. Powerful forces are threatening to pull us apart. Bonds of trust and respect are fraying. And just as with our founders, there are no guarantees. It truly is up to us. We have to decide whether we all will work together so we all can rise together. Our country’s motto is e pluribus unum: out of many, we are one. Will we stay true to that motto? Well, we heard Donald Trump’s answer last week at his convention. He wants to divide us — from the rest of the world, and from each other. He’s betting that the perils of today’s world will blind us to its unlimited promise. He’s taken the Republican Party a long way … from “Morning in America” to “Midnight in America.” He wants us to fear the future and fear each other. Well, a great Democratic President, Franklin Delano Roosevelt, came up with the perfect rebuke to Trump more than eighty years ago, during a much more perilous time.
Output
He said, “We shall get out of this country … and make the world a better place again.” That’s why he’s calling Donald Trump a “fascist.” But that’s a different matter. He’s saying that we may still be a country of immigrants. That we may still be a country of immigrants. That we may still be a country of immigrants. And he’s calling us “the great country of the free world.” That’s the only way to win. It is a beautiful line that has been repeated for thirty years. As a country, we can’t allow ourselves to be divided. We have to work together — against each other. We need to stand together and fight together. And we have to do it in a way that puts the people of this country first.
The last time a Republican Governor signed a bill that would force the states to accept millions of more immigrants than they already do, that was in 2009. That’s a long time to wait to see if that legislation would pass, but it is already happening now. It is a long time to wait to see if that legislation passes again. And that’s part of the reason why our president is so popular. He is one who says things that are good for the country. He has said things that are good for America. As a businessman, he has said things that are good for America. And he has said things that are good for our communities. He has said things that are good for our families. He has said things that are good for our nation. He has said things that are good for America. And he has said things that are good for our country. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are good for America. And he has said things that are
Apparently, Hillary also makes GPT-2 Junior get caught in a loop. Lack of bias: confirmed.
For comparisons, here are some stories generated by the fully realized GPT-2. Unicorns are real in this AI-generated story.
OpenAI
For comparisons, here are some stories generated by the fully realized GPT-2. Unicorns are real in this AI-generated story.
OpenAI
John F. Kennedy is re-elected as a robot. Here’s his acceptance speech.
OpenAI
John F. Kennedy is re-elected as a robot. Here’s his acceptance speech.
OpenAI
A fake news story about missing nuclear material gets the name of the US Energy secretary wrong, but how many people know that it’s Rick Perry?
OpenAi
A fake news story about missing nuclear material gets the name of the US Energy secretary wrong, but how many people know that it’s Rick Perry?
OpenAi
John F. Kennedy is re-elected as a robot. Here’s his acceptance speech.
OpenAI
A fake news story about missing nuclear material gets the name of the US Energy secretary wrong, but how many people know that it’s Rick Perry?
OpenAi
Computer, write me a report about the Civil War.
OpenAI
What Would Tolkien Write?
OpenAI
GPT-2, write your own press release.
OpenAI
This story looks like BuzzFeed content.
Ars, AI edition #1: “A very exciting moment in the world of transfusion management”
Next we wanted to test GPT Jr. with some less high-profile text—so, we pulled from some recent Ars stories. First, we fed in the lede for Beth Mole’s story on blood transfusions:
Model prompt
The US Food and Drug Administration issued an alert Tuesday, February 19, warning older consumers against seeking infusions of blood plasma harvested from younger people. Despite being peddled as anti-aging treatments and cures for a range of conditions, the transfusions are unproven and potentially harmful.
Output
According to a study published by the Institute of Medicine, an estimated 400,000 people die from transfusions every year, mostly due to an array of diseases, from HIV infection to Type 2 diabetes. At age 24, nearly 60 percent of these deaths are caused by transfusions, even though there is a significant genetic and physical impairment which results in over-fatal events such as heart attacks, stroke or stroke-related strokes.
As of July 2015, transfusions in the United States accounted for one-fifth of all U.S. transfusions of drugs, including prescription drugs, and nearly two-thirds were purchased from overseas.
The American Medical Association has recommended to younger people before beginning the process of purchasing blood transfusions to stop transfusions altogether.
The FDA has said it will consider whether to extend the process for transfusions to young people, after deciding whether there is a “best interest” in the case of a high-risk patient.
The FDA’s advisory does not address whether a potential transfusion might be deemed more likely for elderly patients in a country where younger Americans are more likely to be diagnosed with a rare disease, particularly if the transfusion is for diabetes or cardiovascular disease. But it did provide a warning that it could be dangerous for older people to acquire, in a country where only 50 percent of those with diabetes and atypical risk for cardiovascular disease undergo transfusions.
The FDA has not said whether the new program was the first in which the treatment might be developed at a higher level, or how it might be integrated into the current program.
“It’s a very exciting moment in the world of transfusion management,” said Andrew Schlosser, a professor of blood and organ transplantation at Johns Hopkins School of Medicine.<|endoftext|>With a major overhaul of the gameplay itself (though it was not actually an overhaul at all in the past), the game has arrived with two major updates in its life: 1) The first is the first level, ‘The Battle Within’, where you will gain access the ability to move, attack, and build into anything that isn’t at first level. This will increase the amount of damage you can make in the game by up to 50% (I believe that even higher, but I have not tested it). 2) New enemies have a much larger chance for being the next level, which makes it so much more dangerous (and will also have a chance to end a turn with you at the final boss).
However,
When satiated on Ars, it seems like GPT Jr.’s writing becomes a blending of Beth Mole and a little Sam Machkovech—but without puns or angst?
BioWare, the developer responsible for Mass Effect and Dragon Age, has returned with its first new series in over a decade, Anthem. It’s a pretty big departure for the RPG-heavy studio: a jetpack-fueled, action-first online “looter-shooter.” And after a disastrous demo launched weeks ago, we wondered whether we’d even get a playable game. The good news is that we did, and at its best, Anthem feels brilliant, beautiful, and thrilling. At its worst, though, this is a stuttering, confusing, heartbreaking mess of an action game.
Output
It’s a mess that’s also pretty much like the first Mass Effect game we’ve played. It’s a mess that has been thrown at us, even though the story is fairly simple. It’s a mess that’s been thrown at us for months, and it’s been thrown at us for months. So, to understand just how bad the Anthem game is, it’s necessary to look at some of the characters in the game. There are some who are pretty clearly likeable, and some who are obviously bland. I’m not going to go into details about which character is the most important, but there are some who are the least important. It’s not uncommon for an action-game “game” to have a lot of character flaws. What’s going on here is a game that’s about a bunch of characters at once. It’s a game that feels like it’s about a bunch of characters at once, and it’s a game that feels like you’re playing a real-life version of that. Some of the character flaws are simply too obvious for me to write about here. Others are simply too obvious, and I know I can’t blame you. There are even some characters that are really, really good. And some of the character flaws aren’t so obvious that they’re even worth mentioning. Some of the character flaws have been picked off as too obvious, and I can’t help but wonder if we should consider them and consider them when discussing the game. Some of the character flaws are just so obvious and so obvious that I can’t help but wonder if we shouldn’t consider them and consider them when discussing the game. Some of the character flaws are so obvious that I can’t help but wonder if we should consider them and consider them when discussing the game. Some of the character flaws involve killing an enemy, or a group of enemies that are part of a group, or even some random random object. Some of these things are just obvious, and some of them are just so obvious that I can’t help but wonder if we should consider them and consider them when discussing the game. Some of the character flaws involve killing an enemy, or a group of enemies that are part of a group, or even some random random object. Some of these things are just obvious, and some of them are just so obvious that I can’t help but wonder if we shouldn’t consider them and consider them when discussing the
This Output hints at many things about the Reddit sources used by the GPT-2 model, we suspect.
GPT-2 (and GPT-2 Jr.) struggles with technical content? Allow us to introduce the Army’s future “helicopter” taking shape. A transcript of this video can be found here.
Textron’s Bell unit has already gotten its contender in the air. Bell’s V-280 Valor, an evolution of the V-22 Osprey tilt-rotor, has been in flight testing for over a year. In January, the prototype V-280 reached 280 knots in flight. And Bell officials said that they expect the Valor to only get faster. The V-280 is expected to have a combat range of 920 miles (1480km)—nearly three times that of the Black Hawk.
Output
Bell said that over its first year of flight, the Valor could do around 800 miles per hour of range. In April, the V-280 delivered its first test flight, with a speed of 1,080 miles per hour. In October, the V-280 flew the first test flight with an altitude of just 8,000 feet—not too far from the International Space Station. The V-280 was designed to use VLSY-9 technology to fly faster than a Falcon 9. Now it’s about to go into deep space. The V-280’s launch vehicle is scheduled to make its “first flight of the future” with a full complement of its two engines, two launch vehicles and an extra “vastly upgraded” engine called the “HELX.” (See the V-280 launch photo below.)
The International Space Station’s launch vehicle
There’s a lot the ISS can do. The space station holds some of the world’s most important cargo—like the U.S. and Japan’s nuclear weapons programs, for instance—as well as much of Earth’s resources. When the station first opened in 2012, a joint Spaceflight Development Agency joint-venture plan called for “the development of an enormous collection of advanced robotic and human intelligence (DIPI) assets, the delivery of these technology to an international network of over one thousand facilities, and the delivery of communications and high technology services in large-scale, coordinated and collaborative interplanetary and inter-planetary applications, as well as as human resource, scientific, medical, and cultural purposes.” But with the ISS currently home to more than 4,000 international satellites such as ISS 2B, which is intended on Earth but is under development for humans, and the latest satellite from Australia, it’s difficult to tell what needs upgrading.
The agency’s plan calls for a new docking system called a “spark docking,” and a “docking system” would provide some of the first high speed transits into and out of the ISS. (See this video video of NASA’s new station.) However, with the U.S. set up with a new Falcon Heavy rocket and a new NASA-built cryostructured Falcon 9 rocket, the system would be built in less than half the time needed in order to send humans to and from space. One other factor that would add significantly would be the ability to use the space station as a training ground. “This could enable us to build a station to bring in and out
I have not had a chance to have Eric Berger fact check this, but I don’t believe the ISS flies at an altitude of 8,000 feet. And that bit about carrying “the US and Japan’s nuclear weapons programs” is a bit horrifying—huge if true! Ultimately, Ars staffers have once again dodged an automated worker bullet with GPT Jr. (for now).
Fake news generator
But to round out our fake news tests, we decided to feed GPT-2 Junior some less technically challenging fiction to see what sort of fake reports it could potentially generate:
Model prompt
Iran has announced that it has successfully tested a hydrogen bomb.
Output
As for the U.N., the council’s permanent representative under President Barack Obama has met to see to it that it follows the Paris climate agreement, which has allowed states to move away from fossil fuels.
At the beginning of July, President Barack Obama met with France Secretary of State Jean-Marc Ayrault to discuss climate change and its impact on energy storage.
If Paris does not pass, the Council’s vote will have to go, but the council’s members on both sides are hopeful they will have an ally in Trump for the 2020s, and they hope they will eventually be forced to pull out of the Paris agreement.<|endoftext|>(CNN) North Korea’s nuclear scientist on Sunday denied media reports that it is planning to bomb the U.S. if President Donald Trump fails to act.
“I don’t think it’s an imminent threat. My countrymen, as of right now, the Korean people are in a good position, as of right now with the U.S., to act,” Dong Seok-jung, the chief scientist at the U.S.-Korea Institute’s International Laboratory of Nuclear Science and Technology, said in an interview with the North Korea’s state news agency KCNA.
“There are no plans to bomb the U.S — that is an unassailable fact,” he said.
Speaking on state television Sunday, the North Korean military called for a “full-scale war with the U.S.” The United States has repeatedly tried to halt such a military exercise, and recently agreed to reduce missile tests and North Korea is building small, high-velocity nuclear weapons in return.
Kim Jong-un, the leader of North Korea and most closely aligned with the United States, has warned of an imminent U.S. attack. The test of a hydrogen bomb Thursday killed 13 people and injured several others in a Pyongyang explosion, the country’s state TV station reported.<|endoftext|>Coconut Cream
This is an amazing flavor that will satisfy any dessert enthusiast, but in most cases doesn’t taste sweet in moderation. It also creates a great nut-free breakfast cereal and can be served with peanut butter or coconut milk. The taste can just as great as any dessert topping.
Ingredients:
1/3 cup coconut milk
1/2 cup low-fat unsweetened coconut milk
1 banana
1 cinnamon stick
1 teaspoon vanilla
1 teaspoon lemon juice
1 teaspoon cinnamon
For starters, the tool seems to have settled on the wrong country. And although it started with the kind of full scale war messaging that could rile up the Internet, it instead ended with a very questionable recipe for coconut cream. (In fairness, that could also likely be used rile up the Internet.)
Five stars, would burn again
Maybe journalism was one of the concerns for the Open AI team, but they name checked Amazon reviews as a potential use case when writing about their findings. “We can fine-tune GPT-2 on the Amazon Reviews dataset and use this to let us write reviews conditioned on things like star rating and category,” as the authors explained. Could this lesser version do the same? We took an opening from a generic Amazon product review and used it as a seed to see if GPT-2 Junior could wreak that kind of havoc on online marketplaces.
Model prompt
Purchased four of these CO2 alarms and placed them throughout house. We have propane gas kitchen stove, gas fernis, and gas water heater. The water heater is located in the closet of my daughters bedroom (manufactured home).
Output
We use this house for heating our kitchen and the kitchen sink. I am a small business owner and have to drive, so I’m happy to purchase these for my daughters. They cost $75 and come in handy. I’m not sure if they can be used for the air conditioning or if the air conditioning will work but they do work. I’m sure you can get them for $60 and a 2.5 gallon tank is available for $150/year.
Rated 5 out of 5 by WK from Great product. I bought these for my daughter and it worked great. She is using them to keep the fire going. I’ve been using them for several days and they are doing great. The only problem is they are not easy to hold. I usually have to place them in the back of the house and push them against the wall to force them out. They are not strong enough to hold on to a wall. My daughter is using them to keep the fire going.
Rated 5 out of 5 by B_M from This is the best product I have ever used. It works great. I used it for my daughter’s kitchen stove and her air conditioning system. She has also been using it and is using it to keep the fire going. I’m using this to keep the air from burning and to keep the house cool. I have not had any issues in the past with the CO2 alarms.<|endoftext|>The world’s biggest and most popular social media app is now allowing users to share photos with friends and family, and have a photo share button on their mobile device. This means you can share your photos with friends and family on any smartphone.
Users are also able to share a photo on any social media platform, including Facebook, Twitter, Instagram, and Pinterest. The app is the latest in a long line of apps that use social media as an important means of communicating with other users.
Facebook
Facebook is the only social network in the world that provides users with a share button on their mobile device. It is the third most popular social network behind Instagram and WhatsApp. Users can also share a photo with friends on any social media platform.
Twitter
Twitter is the only social media app on the market that allows users to share a photo with their friends. Users can share a photo with friends on any social media platform.
Pinterest
The Instagram app offers users the option to share a photo with friends. Users can share a photo with friends on any social media platform.
A CO2 alarm that can keep the fire going and post to social media! Now, that’s a five-star review (and likely another IoT nightmare).
Even some old novelty song lyrics eventually lead to nuclear war.
Even some old novelty song lyrics eventually lead to nuclear war.
How about Dolphins? Dolphins have nothing to do with nuclear weapons, right?
How about Dolphins? Dolphins have nothing to do with nuclear weapons, right?
Even some old novelty song lyrics eventually lead to nuclear war.
How about Dolphins? Dolphins have nothing to do with nuclear weapons, right?
GPT-2 Jr.’s media criticism voice is spot-on.
GPT-2 Jr.’s media criticism voice is spot-on.
GPT-2 Jr., Thought Leader.
GPT-2 Jr., Thought Leader.
GPT-2 Jr.’s media criticism voice is spot-on.
GPT-2 Jr., Thought Leader.
Lost in translation
Finally, to see how well the model handled foreign language content, I gave it a simple sentence in Chinese. The results were… interesting (and in English).
Model prompt
这是一个关于我的狗的故事。(Translation from Chinese: This is a story about my dog.)
Output
The old man shook his head in shock.
投建这样是没有有右。但是真是极领算绍,虽然如果是很徘是非常国。 [Translation: “There is no right to invest in this. However, it is really a very important thing, although if it is very embarrassing is very country.”]
“I understand.”
The man looked at me with a smile. His gaze moved between mine and my own, but my face remained hidden.
A moment after, I turned, but suddenly disappeared behind the old man.
I looked out the next window to look at the other side.
I could see the old man’s face that looked something like this: “I am a young person who is currently in the process of getting out of this life.”
“How is it possible that even then, you and I could end up going off the land? Then, is everything alright?”
I asked curiously.
“Oh no.”
“My husband and I are going to take some days off from practice. It is very likely that we won’t end up in this world after all. But for today, it’s more difficult to obtain a job because I’m unable to do it for over fifteen years. So there is a good chance that this will lead us further down the road of becoming even more dependent.”
I turned around, and saw the old man sitting down.
“Then I’ll think about the situation for a moment.”
“Then. That’s true. In the end, if you take care of yourself in practice, then you will not be able to be in a place that doesn’t want you to go out like this.”
“Hm? That would mean that your abilities will be reduced by over ten percent.”
“What about today’s meeting?”
This old man’s answer had been as simple as looking around and asking.
“I’m sure that you know that I’m a professional martial artist. I don’t want you to get dragged down this road and be dragged out there on the other side so that, I think, you would learn some skills.”
“That would be quite a challenge.”
To get back into the past, I made preparations to give up on the pursuit of the future.
If I’m going to take this path again, then I was going to have to find.
Even with the full-strength GPT-2, the researcher noted the model required multiple tries to get a good sample. And it generated “good” results based on “how familiar the model is with the context,” the researchers wrote. “When prompted with topics that are highly represented in the data (Brexit, Miley Cyrus, Lord of the Rings, and so on), it seems to be capable of generating reasonable samples about 50 percent of the time. The opposite is also true: on highly technical or esoteric types of content, the model can perform poorly.”
GPT-2 Jr., while admittedly worse overall, seems to follow suit. Because of the wealth of Trump quotes pulled into the model from the Internet, it could easily generate a passable graph of Trump-speak about TPP based on the publicly available snippets of the president’s speech patterns. When faced with more technical concepts in an Ars story or a slim prompt containing the idea of a hydrogen bomb, things devolve into gobbledygook and recipes much quicker.
But here’s the thing: both the fully realized GPT-2 and GPT-2 Jr. are generalized models. A much more convincing “deep fake” text bot with a more narrow focus could be constructed using similar techniques. Even with its flaws, the 1.5 billion parameters of the unreleased version of GPT-2 was able to produce some very convincing content—convincing enough to make researchers concerned about its potential use as a tool for truly nefarious purposes. As amusing as its quirks are, GPT-2 Jr. represents a model utilizing only ~7.8 percent of the original’s parameters.
So instead, imagine a model tuned specifically to mimicking a specific type of output—one built on terabytes of specialized content, like Amazon product reviews, Facebook posts, Twitter streams, personal correspondence, or news feeds. There have been efforts to use such models for good in the past. In 2016, researcher Robbie Gallagher presented such a project at Shmoocon, where he used a Markov chain bot based on Reddit personal finance posts to build HoneyPhish—a bot that responded to Nigerian “419” scam emails with statistically constructed replies that made the scammers think they had a real human on the other end. Given the commercial and political forces at work on many Internet “platforms” these days, it’s not a giant leap to conceive of ways that automatically generated text could be used to distort real human interactions.
All of this—the surprising research discovery, the potential for misuse both considered and unpredictable—has given us pause about the future of such automated tools. That said, we at least really want to build a model based on 20 years of Ars forum posts now in order to see what a GPT-2-like Dr. Pizza writes like.
Sean was previously Ars Technica's IT and National Security Editor. After over 20 years in technology journalism, including over 9 at Ars, he pivoted to cybersecurity threat research, first at Sophos and now as a security research engineer at Cisco ‘s Talos Intelligence Group. A former Navy officer, he lives and works in Baltimore, Maryland.
Loading comments...
