May I make a catalog script, mods?
1. May I make a catalog script, mods?
2. May I use /prog/ for this?
3. What do mods let KC be?
1. About Catalog Script.
I feel KC has 3 matters which may be added. Catalog, floating post form and function of showing 'You' in the reply which was made to me. I already made a script for 'floating post form', so now I'll try to make a script for generating a catalog. I mean catalog like those in 4chan or other most of all chans have.
However, lacking of catalog or OP search function is also a part of characteristic of KC anymore. It force us read or see many of thread than other chans. Therefore, people who don't want to read or see other thread will leave here. People who don't want to read longer sentences will also leave here, because why they hate longer sentences comes from their poor reading skill, which consumes more time than well-educated expert. Repeatedly, KC forces us to read more sentences to find or join a thread, this make a pressure to remove people who don't have enough reading skill.
But lacking catalog function scatters residents. We can't find the thread we want to participate easily. Sometimes we made a multiple thread just because we couldn't find existence of similar thread. And personally I think time for searching thread is waste of time. I want to consume time for considering or making a reply, not for searching.
So, I want to make a catalog. May I make it although I know the catalog have both good and bad faces.
3. Still I wonder what mods let KC be. Now I don't have an idea because I haven't seen many threads so far because of lacking catalog. This is one reason why I want a catalog.
Anyway system character and criteria of moderation are key of chans' character. And no catalog is big. If mods don't want to make a catalog, I shouldn't make it.
If you want, I can add a tag for search. For example, pre-defined keywords for search will become a tag if people want to use it for keyword, like #news and #IWO. If the script download the keywords from KC, you can control it at any time.
I'll add graph of post/thread ratio. You can see KC's ratio is low, because people are scattered.
(Number of new threads are not acquired well in some pic.)
KC/int/ and KC/b/.
You can see average posts in a thread is about 20.
(The graph contains the down time of yesterday.)
4chan /int/, /a/, /b/ and /jp/.
Average posts/thread is about 60-80.
(Number of thread is not acquired well in graph of /b/.)
2chan.net/b/ and 8chan/v/.
(Number of threads is not acquired well in 2chan.net/b/. 8chan doesn't contain enough points.)
A client sided catalog is a bad idea, because every user had to scrape all 20 pages on every refresh, which could lead to a high load on the server.
There is an experimental version of a website based catalog with regular refresh intervals and a basic search function.
What is the name of the script?
> which could lead to a high load on the server.
No. Each page's HTML are about 35KB, so it invokes 35*20=700KB transaction. This is equivalent of a middle sized image, not so heavy.
You are forgetting the network overhead, so a single transferred page inflates to ~160KB on the wire. Try it for yourself with tcpdump/wireshark.
That times 20 for every user every X minutes.
And then you still have to fetch the pages and parse them via js. One page takes around 2 seconds, so >30 sec for the whole set unparallelized + 10 seconds parsing via libxml2, a C library, js will be slower of course. Not exactly a fluent experiences, especially is you want to refresh the catalog often.
As stated it's not directly a (user)script, but a webpage.
> As stated it's not directly a (user)script, but a webpage.
> implying you are one of administrators
Why don't you release it, or make an official(server-side) catalog? That is the best solution technically.
And I checked consistency of transferred size between WireShark and developer's tool in Chrome. They are consistent.
Now I'll download only required things, because I'll use XMLHttpRequest(), which don't interpret HTML. Pic.1 shows that, I got 0-11 pages, 35-50KB*11pages, they aren't inflated, that's all.
> Why don't you release it
It thought I already linked to it, my bad: https://blauert.indus.uberspace.de/int.html
>or make an official(server-side) catalog?
To the author it was more of a little hack to play with a programming language, than a serious project. But the complete source code is available, so anyone with a http server can set it up.
Thanks, I'll try it.
BTW, you know what technique the site uses, this means you are the man or near the man who wrote this script, right? Then, may I ask questions for you if I encounter problems?
Do you know the refresh rate of the site?
> 15 minutes for /b/ and 30 minutes for /int/.
Why? 5 minutes for /int/, please.
> Sure, there is also an email address listed on the page.
Sorry, I can't find it, but now I can contact with you here, this is OK.
Now I made a prototype of catalog.
If you want to see, I'll upload it.
So far, it is quite light, because I implement a software-cache in the script. But often it crashes when I drag the window to move with catalog shown.
I uploaded it since previous version has a middle level bag.
If you are interested, read >>/jp/36393 and use it, but it contains other problems.
Give those guys a goddamn catalog already or at least an API.
We will add an official catalog this year, please don't build anything that pointlessly increases server load, because having an arms race trying to get rid of it would be a waste of everyone's time. Thank you.
Sorry，I'll never stop developing it until you release it. And I'll implement cross-board(working now) and cross-site catalog, so it have a point, which will remove the barrier between sites.
>Sorry，I'll never stop developing it until you release it.
Good to know who I'm dealing with then.
>As for the server load, I mentioned above, and I checked it's going well.
It's beyond your means to check, really. Anyway, expect me to break it or block people who I suspect are using it if it seems to be causing problems. Thanks for your cooperation.
> It's that you might be downloading a lot of pages that are normally not cached and will therefore have to be generated on the fly.
Then, cache it please. Just 20 pages, 1MB. Lower pages allow us to use longer lifetime. About this point, I felt you should adopt longer lifetime at least for the banner. I saw 200 instead of 304 many times just because of revised banner.
And you said the bottle neck is not network, but CPU usage, that is beyond me, that is really your business.
Dude, if that's your position and you have already declared that you more or less don't give a fuck even after I have asked you nicely not to do anything that might put extra load on the site, can you please give me a reason not to ban your silly threads where you advertise this thing outright?
Because if you can't, that's exactly what I'm going to do.
> Then, cache it please.
As far as I know that would entail either changes to the caching model or writing 21 pages of html per post instead of just two.
Please just provide some APIs for developers. Like a cached RSS-Feed with all Threads, Metadata about the images (md5 hash) in html and feed, etc.
This would improve userscripts while reducing the load caused by these scripts.
I guess it wouldn't hurt to provide a longer RSS feed (right now it's just 20 threads on /int/).