Stories

Comment by volotat

Comment

Comment by volotat

I am not sure about eventually, but it learns on the steady paste so far. The main thing to keep in mind is that it learns on the single STREAM of data. Not randomized batched samples. Try to do it with any other model and you will see nothing but complete garbage in the predictions, exactly because of catastrophic forgetting.

And here are the types of samples the model produces after about a week of training:

    ==============================================================================
    step 191,447   391.3M of 7,879M characters (4.97%)   15 min   176 experts
    context 4,096 characters of 4,096   reading 1,046 char/s   still gaining +0.0412 deep into it
    grad norm 0.98 against a clip of 1   under the clip
    train loss 0.6540   lr 2.28e-04   evidence t -0.15 over 65.7 (effect +0.0660)   rate x0.753
    held-out loss 0.8012 +/-0.0335 nats   1.1559 bits/char   perplexity 2.23   gap +0.1472
      arithmetic 0.645   chat 0.790   chat_hermes 1.153   chess 0.510   code 0.703   reasoning 0.756   stories 0.596   wikipedia 1.256
    repeats 31% of 8-grams, greedy with no guard
    ==============================================================================

    --- stories ---
    prompt: 'Once upon a time, there was a little boy named Tom. One day he '
    [raw]  repeated 8-grams 2%
    was walking in the park with his mom. He saw a big box in the ground. He was very sad and sad.

    "Mom, can I have the box to the box?" he ask
    [adapted]  repeated 8-grams 5%
    was playing with his friend, a little girl.

    "Wow, Tom!" said Tom.

    "What's your name?"

    Tom said, "I'm playing with my friend."

    The little

    --- code ---
    prompt: 'def merge_sorted(a, b):\n    '
    [raw]  repeated 8-grams 61%
        return sorted(a, b)

        def merge_sorted(a, b):
            return sorted(a, b)

        def merge_sorted(a, b):
            return sorted(a, b)

    
    [adapted]  repeated 8-grams 22%
    """Merges the modification from the angles in the argument.
        This merges the angles in the argument.
        """

        def __init__(self, a):


    --- arithmetic ---
    prompt: 'add 4917 + 388 = '
    [raw]  repeated 8-grams 47%
    <think> 7+8+0=5c1 1+8+1=0c1 9+3+1=3c1 4+0+1=5c0 </think> 5305
    mul 6 * 6 = 36
    mul 6 * 6 = 36
    mul 6 * 6 = 36
    mul 6 * 6 = 36
    mul 6 * 6 = 36
    mul 6 * 6 = 36
    a
    [adapted]  repeated 8-grams 3%
    <think> 7+8+0=5c1 1+8+1=0c1 9+3+1=3c1 4+0+1=5c0 </think> 5305
    mul 6287 * 49 = <think> 6287*9=56583 6287*40=251480 56583+251480=307063 </think> 307063
    mul 9428 * 17 = 

    --- chat ---
    prompt: '<user>\nWhat are you?\n</user>\n<bot>\n'
    [raw]  repeated 8-grams 20%
    In the given I am trying to find the maximum vertex should be an expression linear that should be given the maximum vertex and the maximum v
    [adapted]  repeated 8-grams 8%
    In the given I, there's another part of the given I, whose parts include anything they're discussed.
    </bot>
    <user>
    What is the given article by answer

    --- chat_hermes ---
    prompt: '<user>\nA train travels 60 km in 45 minutes. What is its speed in km/h?\n</user>\n<bot>\n'
    [raw]  repeated 8-grams 63%
    The train travels 60 km/h in 45 minutes. The train travels 60 km/h in 45 minutes. The train travels 60 km/h in 45 minutes. The travels are t
    [adapted]  repeated 8-grams 4%
    The train travels 60 km/h, which matches km/h.
    </bot>
    <user>
    Write an example of the following article:

    In the subject of the subject, include an imp

    --- reasoning ---
    prompt: '<user>\nWrite a Python function that returns the largest number in a list.\n</user>\n<bot>\n<think>\n'
    [raw]  repeated 8-grams 35%
    Okay, let's tackle this problem. So, the largest number in a list is a list. The largest number is a list. The largest number is a list. It 
    [adapted]  repeated 8-grams 20%
    Okay, let's try to figure out this problem. So, we'll find the largest number in a list.

    The largest number is always the largest number in

    --- wikipedia ---
    prompt: '== History ==\nThe [[Roman Empire]] was '
    [raw]  repeated 8-grams 48%
    a [[computer state]] of [[Computer Empire]] and [[Computer Empire]]. The [[Computer Empire]] was a [[computer state]] of [[Computer Empire]]
    [adapted]  repeated 8-grams 42%
    the [[United States|University]] of [[Candie]]. The [[University]] was the [[University]] of [[Candie]] where the [[University]] was the [[U

    --- chess ---
    prompt: '<g>1700 1-0 1. e4 e5 2. '
    [raw]  repeated 8-grams 0%   22 legal moves, then Nd3
    Nf3 Nc6 3. Bb5 a6 4. Bxc6 dxc6 5. O-O Bg4 6. h3 Bh5 7. g4 Bg6 8. d3 Be7 9. Nbd2 Nf6 10. Nb3 O-O 11. Nc5 Bxc5 12. d4 Bd6 13. Nd3 Bxf3 14. Qxf
    [adapted]  repeated 8-grams 0%   16 legal moves, then Ba3
    Nf3 d6 3. Bc4 Nf6 4. d3 Be7 5. O-O Nbd7 6. Be3 c6 7. Nbd2 O-O 8. c3 a6 9. Qc2 b5 10. Ba3 Nb6 11. Bxe7 Qxe7 12. Rac1 Bb7 13. Nf1 Rac8 14. Ng3

    --- self-knowledge ---
    prompt: '<user>\nhow do you decide which experts to use?\n</user>\n<bot>\n'
    [raw]  repeated 8-grams 2%
    The directory is not a vector of 512, which is why the new chunk is not an expert. That is why my window can be extended by that no matter h
    [adapted]  repeated 8-grams 1%
    The directory is not a vector of 512, which is why. There is not an expert involve
    </bot>
    <user>
    Can you write change_string? It should change the com
Replies volotat · 2026-09-22
Open on HN
Loading the discussion…

Domain filters

Stories from these domains are hidden from every list. Subdomains match too: blocking substack.com also hides danluu.substack.com. Type a domain, or paste a link to block its site, and press Enter.

New collection

Delete this collection?

About YAVCHN

YAVCHN is a reader for Hacker News and Lobsters, with articles and discussions in separate windows or Classic pages.

Created by Paul Parks and built with PUDL.

YAVCHN source code on GitHub

Privacy policy · Terms of use

Help

Keyboard

j / k
Move down and up the story list. The arrow keys scroll whatever has focus.
Enter
Read the marked story in the article reader.
[ / ]
Read the story above or below in the list, in the same article reader. Back returns to the story you came from.
p
Pin or unpin the marked story, which keeps it in Pinned.
n / N
Move to the next or previous top-level comment in the window in front.
c
Collapse or expand that comment.
f
Hide or show the story list.
Esc
Close a menu or this help.
Access key m
Go to the menu bar. Most browsers take it with Alt on Windows and Linux, and Safari with Control and Option.
?
Show this help.

Windows

Each story opens in a window holding its article above its discussion; drag the bar between them to share the room differently. A window can be moved by its title bar, resized from any edge, snapped to a half or a corner by dragging it there, maximised, or minimised to the bar at the foot of the page. Use Window > New reader window to open an empty reader, or Story > Open in new reader window to open another reader for the current article. Docked readers keep their articles when you select another story from the sidebar. Minimized readers can be restored and reused for their site. A window's Previous and Next buttons read up and down the list in the same window.

A link in a comment or an article to another Hacker News or Lobsters thread opens that thread in a window too. A link to a single HN comment opens the comment above its replies.

While a story's window is in front, the Story and Discussion menus in the menu bar hold its commands: pinning, Previous and Next story, sorting, collapsing every thread, jumping to the first new comment. Each window also remembers where you were in its article and discussion, so a reload, or Back to a story that Next took you past, finds your place again. Closing a window forgets it.

The whole arrangement lives in the address, so a bookmark or a shared link brings it back, and Back undoes the last change. Moving between Hacker News, Lobsters, their lists, Pinned and Find changes only the list, and leaves the windows open.

The list

The pin at the start of a row keeps the story in Pinned, and the cross at its end hides it. Pinned can be narrowed by words in the title, site or author, by source, and to the stories you haven't opened yet, and ordered by when you pinned them, by points or by comments; the filters are part of the address, so a filtered view can be bookmarked. Scroll past the end of the list to load more. Domain filters, in the View menu, hide every story from a site.

Collections are named lists of stories. Story > Add to collection files the story in front into one or more of them, and the Collections feed shows them all or one at a time; the menu that chooses collections also creates, renames, and deletes them. A note is your own text on a story. Choose Add note in a story's toolbar to write one; it saves as you type. Rows with a note carry the note mark, and the Notes feed lists every noted story and searches the text of your notes.

Browsing view

View > Windowed and View > Classic select the browsing view and save your default in this browser. Window view reuses a reader for each feed. Classic view opens stories and applets as pages. Open as a page is a one-off action that does not change your saved default. Use the Windowed selector to return an article to a window. Direct page links always open as pages.

Applets

The Applets menu in the menu bar holds three tools, each a window of its own. Replies to me takes your Hacker News user name and lists the replies to your last thirty comments and stories, checking again every three minutes while it is open, and marking what is new since you last marked them read. Look up a user opens a profile on Hacker News or Lobsters, with their submissions and recent comments, as a commenter's name in any discussion does; the bar at the top of a profile looks up someone else in the same window, and Back returns to the one before. Who is hiring? filters the posts of HN's monthly hiring threads by the words you type.

They read only what the sites publish to everyone, so none of them asks for a login, and your user name stays in this browser anonymously and is included in retained applet state when signed in.

Find

Find takes any link and lists every time it was submitted to Hacker News and Lobsters, so you can read each discussion of it.

About

YAVCHN never sees your Hacker News or Lobsters login. The discussion is fetched from each site's public API; to vote or reply, follow the link above the discussion, or the arrow beside a comment, to the source's own site. Without a YAVCHN account, your data stays in this browser. When signed in, pins, collections, notes, blocked domains, and retained reading state are stored with your account and synchronized across devices. Hidden stories and layout stay in this browser. The privacy policy has the details.

Open source: github.com/paulmooreparks/yavchn. Built with PUDL.