TwoKinds [of] data
Moderator: Moderators
-
- Apprentice
- Posts: 102
- Joined: Wed Jul 10, 2019 1:52 pm
- Location: Finland
- Fav. Twokinds Character: Keith
Re: TwoKinds [of] data
Could you consider doing the very first graph again in a different way? Currently there is so much data crammed into so little horizontal space that it is impossible to see anything else than the low and high extremes. Maybe you could make a simple plot showing how many updates there were each year or half-year?
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
Yes I know the first graphs were a bit cluttered. ,Sorry about that
Unfortunately I lost my code and the data I used to make that. I am working on rewriting everything but it will take a while. Sorry
Unfortunately I lost my code and the data I used to make that. I am working on rewriting everything but it will take a while. Sorry
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
So after two weeks or rewriting all my scripts and rebuilding the dataset from scratch i managed to finally be able to get a new graph.
Spoiler! "Pages Per month"
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
- Neutral Smith
- Templar GrandMaster
- Posts: 673
- Joined: Mon Feb 24, 2014 5:32 pm
- Location: Neitherlands
- Fav. Twokinds Character: Saria's father
- Contact:
Re: TwoKinds [of] data
I think the interval between pages is more interesting than the amount of pages per month.
If you have a page on day 1-8-15-22-29 you have 5 pages in a month with a 7 day interval.
If you have a page on day 6-12-18-24-30 you also have 5 pages, but this time the interval is 6 days.
If you have a page on day 1-8-15-22-29 you have 5 pages in a month with a 7 day interval.
If you have a page on day 6-12-18-24-30 you also have 5 pages, but this time the interval is 6 days.
If it moves but shouldn't: Duct tape. If it doesn't move but should: WD-40. With a hammer everything fits, and if it still doesn't fit: bigger hammer. If it can't be fixed with a hammer it's an electrical problem.
-
- Apprentice
- Posts: 102
- Joined: Wed Jul 10, 2019 1:52 pm
- Location: Finland
- Fav. Twokinds Character: Keith
Re: TwoKinds [of] data
The thing is, you couldn't actually see the individual intervals in the first graph. It was way too cluttered. Now it is possible to make observations about how update frequency has changed over time. For example, it seems that the rate is in slow decline, though it fluctuates a lot.Neutral Smith wrote: ↑Sun Aug 02, 2020 6:42 am I think the interval between pages is more interesting than the amount of pages per month.
If you have a page on day 1-8-15-22-29 you have 5 pages in a month with a 7 day interval.
If you have a page on day 6-12-18-24-30 you also have 5 pages, but this time the interval is 6 days.
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
You are kinda of right. The think is at the beginning, like ten years ago, Tom posted twice a week. Then around page 450 he took a small rest, less than 2 months and changed his schedule to once a week. So technically yes he is posting less often but that is simply because hos schedule changed. That is why it is importan how you present data as that migth give wrong impressions.Yastreb wrote: ↑Mon Aug 03, 2020 3:16 pm
The thing is, you couldn't actually see the individual intervals in the first graph. It was way too cluttered. Now it is possible to make observations about how update frequency has changed over time. For example, it seems that the rate is in slow decline, though it fluctuates a lot.
But you gave me this idea to graph the same histogram for different time spans: quarterly, every semester and yearly:
Spoiler!
If you only look at the number of pages posted yearly it looks like there is a slow decline on frequency but if you zoom in a little you realize there were simply more post at the beginning and nowadays the post frequency has maintained.
Also per popular I decided to re-make the graph of number of pages per month.
Spend a lot of time trying to figure out a way to make them more clear. Finally came to the conclusion that the best way was to split the graph into different section. Effectively
making a very very long composite graph. For to make it wasier to read I made it a vertical graph, and increased color contrast. On the vertical axis is the page number and the horizontal axis is the number of days between each page.
Spoiler! Warning very long
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
-
- Apprentice
- Posts: 102
- Joined: Wed Jul 10, 2019 1:52 pm
- Location: Finland
- Fav. Twokinds Character: Keith
Re: TwoKinds [of] data
Thanks a lot! These are very clear. But are the x-axes of the first three graphs in your latest post labeled right? They seem to have data from the year 2021.
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
You know what? You are right the way I grouped days for the counting was at the end of the year, hence i get the last data point on December 2020 which in the graph is indistinguishable from 2021. Fixed the error and reuploaded, also changed the graph style to increase contrast and hopefully make it more clear.
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
You know i had this idea a couple months weeks ago. We have this old thread about our favorite character from the comic. Unfortunately the poll is rather old and has probably less than a hundred votes in total so really not much data in there and on top of that is probably biased.
But it got me thinking, how could I gauge the popularity of the comic characters in the most objective way possible? The problem here is how I can you measure that without asking everyone which is rather inconvenient. After a couple weeks days of mulling over it I had a realization: "The highest form of flattery is imitation" If I could somehow measure how much fan-art exists of each character that is a good indicator of how popular each character is. Moreover Tom sketches are direct suggestions by his biggest fans, so if I threw those into the mix too i could get a good idea of who is Twokinds most popular character.
Now how to measure that? My idea was to crawl the internet and find all Twokinds related artwork, made by Tom or otherwise. That is harder than it sounds. the web is large and probably focusing my search is a better idea. Tried consulting Patreon first but it only exposes all content to paying patrons, as it should. Also tried DeviantArt but it is by far too large for this and a pain to scrape, besides it not so popular anymore so it is likely to miss a lot of stuff. Eventually i remembered came across a site that likely archived all the info i needed and had a simple API I could consult. I cannot tell you which site as it is "uncompliant" with the forum rules but rest assured it is an authoritative source.
So after all that verbiage here is what i got:
First graph shows how many post I could find where each character appears, whether it was create by Tom or otherwise. And the second the average score, likes - dislikes, each character gets on every pieces they appear on.
So what can we conclude from this? Personally, that was an inordinate amount of work for 2 little graphs with not that many insights. Thankfully I finished this already and i can cross it out of my to-do list. And fortunately the infraestructure i laid down for this can be reused for other ideas.....
But it got me thinking, how could I gauge the popularity of the comic characters in the most objective way possible? The problem here is how I can you measure that without asking everyone which is rather inconvenient. After a couple weeks days of mulling over it I had a realization: "The highest form of flattery is imitation" If I could somehow measure how much fan-art exists of each character that is a good indicator of how popular each character is. Moreover Tom sketches are direct suggestions by his biggest fans, so if I threw those into the mix too i could get a good idea of who is Twokinds most popular character.
Now how to measure that? My idea was to crawl the internet and find all Twokinds related artwork, made by Tom or otherwise. That is harder than it sounds. the web is large and probably focusing my search is a better idea. Tried consulting Patreon first but it only exposes all content to paying patrons, as it should. Also tried DeviantArt but it is by far too large for this and a pain to scrape, besides it not so popular anymore so it is likely to miss a lot of stuff. Eventually i remembered came across a site that likely archived all the info i needed and had a simple API I could consult. I cannot tell you which site as it is "uncompliant" with the forum rules but rest assured it is an authoritative source.
So after all that verbiage here is what i got:
First graph shows how many post I could find where each character appears, whether it was create by Tom or otherwise. And the second the average score, likes - dislikes, each character gets on every pieces they appear on.
Spoiler! Post and score per character!
Spoiler! Analysis
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
-
- Apprentice
- Posts: 102
- Joined: Wed Jul 10, 2019 1:52 pm
- Location: Finland
- Fav. Twokinds Character: Keith
Re: TwoKinds [of] data
It took me few seconds to guess what site you are talking about, and after cross-checking the data I now know for sure.Technic[Bot] wrote: ↑Sat Dec 05, 2020 6:10 am Eventually i remembered came across a site that likely archived all the info i needed and had a simple API I could consult. I cannot tell you which site as it is "uncompliant" with the forum rules but rest assured it is an authoritative source.
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Generative Adversarial Keidran
So as you may have figured by now I have a soft spot with artificial intelligence and neural networks. These are the darlings of today's computer science research and as such there is basically a neural network for anything. So I was looking around some youtube videos and found pixel2Style2pixel or pSp a deep learning model (Generative Adversarial network) for face encoding. I will spare you the details, a simple explanation can be found here. suffice to say this model is capable of generating realistic faces faces.
"That is all nice and good but why are you telling us this?" I hear you say, well type. Realistic face generation is not a relatively new idea and has already been done by NVIDIA for example. But controlling the type and output of the face was kinda of hard. pSp does some clever augmentation to allow you finely tune and control the face it creates at the output.
"Get to the point" Patience I am about to get there. What really sold me to try this thing is it is able to take "sketches" and generate a corresponding face.
"Tom has a lot of sketches..." Exactly!
I wondered if I could take some of Toms sketches and use this model to output a "real life" version of, say Flora, moreover, due to the model limitations human Flora. Basically I am using this to get a real life version rendition of our favorite Twokinds characters, converting all Keidran into human in the process. Who needs artistic talent magic when you have cutting edge artificial intelligence!*
So what I did was take some of Tom sketches, cropped the head, model does not do bodies, and shoved them into my artificial intelligence. It is important to not this is not the specific input the neural network likes. It was trained with sort of "pencil sketches". And it was trained using pictures of exclusively celebrities so it is biased towards pretty people.
Anyhow I you are still with me after all that rambling here is the results. I spoilered ans sectioned it to keep this post from being too long.
First our baseline this is what the network expects to use as a "sketch" and an example output. For all images, left is input and right is output, or whatever the neural network vomitted.
Now let show some real stuff
So something a bit more interesting.
I hear you like sneps
The Magi Brothers
Our favorite shapeshifte:
The pair that could not be:
And finally Trace!
And now some general discussion.
As I mention before the AI is not used to seeing this type of sketch. And it is biased toward artist and celebrities faces. That explains most of the "errors" the network never saw anime hair, of which Tom characters have a lot. So it settles for a messy unkept hair. Of course if it had been trained on Sephiroth Cloud and other FF characters we would have got a better results. I imagine some preprocessing could improve the result, removing color adjusting aspect ratio and whatnot but again this is stress testing the AI there is just so much i can do to improve it without retraining the whole thing again.
Also yes with some effort you can tune it to change hairstyle, skin color and gender of your own pictures, this is only one toy application of this AI.
Finally have the imgur link so you can view this at your leisure. Plus some other results that did not fit the post
I hope you enjoyed this little edition of me playing around with twokinds and neuralnetworks. Think of it as sort of Christmas present, hope at least a made you laugh a bit or you learned something new
* I have always wanted to be able to draw, but i don't have any artistic skils...
"That is all nice and good but why are you telling us this?" I hear you say, well type. Realistic face generation is not a relatively new idea and has already been done by NVIDIA for example. But controlling the type and output of the face was kinda of hard. pSp does some clever augmentation to allow you finely tune and control the face it creates at the output.
"Get to the point" Patience I am about to get there. What really sold me to try this thing is it is able to take "sketches" and generate a corresponding face.
"Tom has a lot of sketches..." Exactly!
I wondered if I could take some of Toms sketches and use this model to output a "real life" version of, say Flora, moreover, due to the model limitations human Flora. Basically I am using this to get a real life version rendition of our favorite Twokinds characters, converting all Keidran into human in the process. Who needs artistic talent magic when you have cutting edge artificial intelligence!*
So what I did was take some of Tom sketches, cropped the head, model does not do bodies, and shoved them into my artificial intelligence. It is important to not this is not the specific input the neural network likes. It was trained with sort of "pencil sketches". And it was trained using pictures of exclusively celebrities so it is biased towards pretty people.
Anyhow I you are still with me after all that rambling here is the results. I spoilered ans sectioned it to keep this post from being too long.
First our baseline this is what the network expects to use as a "sketch" and an example output. For all images, left is input and right is output, or whatever the neural network vomitted.
Spoiler! Baseline
Spoiler! Flora
Spoiler! Kat!
Spoiler! Adira the bartender
Spoiler! Nat and Zen
Spoiler! Raine
Spoiler! Laura and Keith
Spoiler! Mrs Legacy
As I mention before the AI is not used to seeing this type of sketch. And it is biased toward artist and celebrities faces. That explains most of the "errors" the network never saw anime hair, of which Tom characters have a lot. So it settles for a messy unkept hair. Of course if it had been trained on Sephiroth Cloud and other FF characters we would have got a better results. I imagine some preprocessing could improve the result, removing color adjusting aspect ratio and whatnot but again this is stress testing the AI there is just so much i can do to improve it without retraining the whole thing again.
Also yes with some effort you can tune it to change hairstyle, skin color and gender of your own pictures, this is only one toy application of this AI.
Finally have the imgur link so you can view this at your leisure. Plus some other results that did not fit the post
I hope you enjoyed this little edition of me playing around with twokinds and neuralnetworks. Think of it as sort of Christmas present, hope at least a made you laugh a bit or you learned something new
* I have always wanted to be able to draw, but i don't have any artistic skils...
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Re: TwoKinds [of] data
Pretty spooky, sis.
- Technic[Bot]
- Grand Templar
- Posts: 1246
- Joined: Sat Jan 27, 2018 9:48 pm
- Location: México
- Fav. Twokinds Character: Raine!
- Contact:
Re: TwoKinds [of] data
RETRACTED
After thorough analysis i realized my methodology was mistaken. I apologize for this, if you are still interested please scroll a couple of post to see corrected versions of the graphs presented here.
I am leaving the original version in place because i think it shows some interesting stuff, not on the data, that is wrong but on our biases.
So today i was Inspired!
I was lurking this social network thing young whippersnapper like to use these days. They called it reddit, once I finally managed to figure out how to navigate that mess of comments in a somewhat efficient manner, I do not really understand the design of that thing. I found there is a page dedicated entirely to twokinds, well 2 but we will ignore the second one. It is not a particularly large community, although is far larger than us, which is not saying much. But it has some activity. A post that caught my eye was: Hope linking reddit works Horny and Normie sketches illsutrated. Had to bust out my "Dictionary of youthful slang" to figure out what the title meant, but fortunately the description as much more clear. The author counted Twokinds sketches from 2020 and classified them as either questionable, if it had only characters posing seductively, or safe if the characters were doing anything else.
"That is interesting" I thought. However that piece is limited to only sketches made last year and had to be counted an classified manually. Then i remembered, did I managed to automatically scrape all known Tom sketches like a couple months ago? Could i do something similar? Of course! Not only that but since I do not have to manually count each sketch and the dataset I use already has a rating for each piece I can effectively graph this over time.
So I refreshed my dataset I went and scraped everything made by Tom, no fan art only stuff attribuible to Tom. Since we have records, around 2009, to this day. And classified either as Safe or Questionable, graphed both the number of pieces we have for each category and also the proportions of each for every year. Fortunately i could repurpose everything I had done before, else this may have took weeks, and finally produced these charts:
As a disclaimer I did not labelled the dataset and sometimes i did not quite agree with it, such as the latest sketch, Built in duster , is considered safe but i find it kinda questionable myself. Anyhow I just used it as is this keeps me from inadvertently biasing the data.
Discussion:
In general it seems Tom has always drawn some suggestive content but there is a marked increase of the number of risque material from 2015 and over, since then the number of suggestive sketches has increased steadily over the years nowadays around 80% of the pieces have suggestive themes. There is also a marked increase in the number of sketches from 2016 and over, not sure why, maybe that is the year he set up his Patreon? I think this speaks more of us the audience than of Tom, after all the sketches are proposed and voted by paying Patreons not himself.
This was fun! I may even copy-pasted this to the Reddit if i can figure out how it works that is...
After thorough analysis i realized my methodology was mistaken. I apologize for this, if you are still interested please scroll a couple of post to see corrected versions of the graphs presented here.
I am leaving the original version in place because i think it shows some interesting stuff, not on the data, that is wrong but on our biases.
So today i was Inspired!
I was lurking this social network thing young whippersnapper like to use these days. They called it reddit, once I finally managed to figure out how to navigate that mess of comments in a somewhat efficient manner, I do not really understand the design of that thing. I found there is a page dedicated entirely to twokinds, well 2 but we will ignore the second one. It is not a particularly large community, although is far larger than us, which is not saying much. But it has some activity. A post that caught my eye was: Hope linking reddit works Horny and Normie sketches illsutrated. Had to bust out my "Dictionary of youthful slang" to figure out what the title meant, but fortunately the description as much more clear. The author counted Twokinds sketches from 2020 and classified them as either questionable, if it had only characters posing seductively, or safe if the characters were doing anything else.
"That is interesting" I thought. However that piece is limited to only sketches made last year and had to be counted an classified manually. Then i remembered, did I managed to automatically scrape all known Tom sketches like a couple months ago? Could i do something similar? Of course! Not only that but since I do not have to manually count each sketch and the dataset I use already has a rating for each piece I can effectively graph this over time.
So I refreshed my dataset I went and scraped everything made by Tom, no fan art only stuff attribuible to Tom. Since we have records, around 2009, to this day. And classified either as Safe or Questionable, graphed both the number of pieces we have for each category and also the proportions of each for every year. Fortunately i could repurpose everything I had done before, else this may have took weeks, and finally produced these charts:
As a disclaimer I did not labelled the dataset and sometimes i did not quite agree with it, such as the latest sketch, Built in duster , is considered safe but i find it kinda questionable myself. Anyhow I just used it as is this keeps me from inadvertently biasing the data.
Spoiler! Number of safe and questionable sketches over time
Spoiler! Ratio of safe and questionable sketches over time
In general it seems Tom has always drawn some suggestive content but there is a marked increase of the number of risque material from 2015 and over, since then the number of suggestive sketches has increased steadily over the years nowadays around 80% of the pieces have suggestive themes. There is also a marked increase in the number of sketches from 2016 and over, not sure why, maybe that is the year he set up his Patreon? I think this speaks more of us the audience than of Tom, after all the sketches are proposed and voted by paying Patreons not himself.
This was fun! I may even copy-pasted this to the Reddit if i can figure out how it works that is...
There are three things that motivate people: Money, fear and love.
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
Links to my ramblings:
Twokinds [of] data
PhpBB in the age of facebook
If you are new to this phpBB thing:
BBCode guide
-
- New Citizen
- Posts: 38
- Joined: Sat Jan 23, 2021 3:21 pm
- Fav. Twokinds Character: ---
Re: TwoKinds [of] data
Exactly what I was thinking.Technic[Bot] wrote: ↑Mon Mar 29, 2021 4:56 am I think this speaks more of us the audience than of Tom, after all the sketches are proposed and voted by paying Patreons not himself.