network subsystem

Overview

The OpenKore network subsystem roughly consists of the following classes:

The Network class

This is the connection manager. It manages the TCP/IP socket connection with the server, and handles things like connection, disconnection, sending data to the server, receiving data from the server, etc. But it doesn't do much else.

Schematically, it looks like this:

Upon connection to the server, it creates two objects:

• A message parser object, which is of class Network::Receive::ServerTypeX. Whenever a message is received from the server, that message is passed to the message parser.
• A message sender object (not seen in the above diagram), which is of class Network::Send::ServerTypeX. This is used for sending messages to the server.

There are several implementations of Network class: Network::DirectConnection, Network::XKore and Network::XKoreProxy.

The Network::MessageTokenizer class

This is a tokenizer class. It extracts discrete server or client messages from a byte stream passed by the connection manager. But it doesn't do much else.

The Network::PacketParser class

This is a base message parser class. It parses messages passed by the connection manager into hashes with message data, as well as does reverse operation - generates messages from hashes with message data. But it doesn't do much else.

Afterwards, parsed messages are handled by built-in handlers in following classes and, with hooks, by plugins or other modules.

Additionally, there is API for modifying or dropping messages to alter any further processing.

The Network::Receive, Network::Send and Network::ClientReceive classes

There classes are descendants of Network::PacketParser class.

Network::Receive and Network::Send are parser helper classes. They contain parser helpers which serve as a workaround in the absense of the capable parser subsystem.

Network::Receive and Network::ClientReceive are message handling classes. They contain built-in handlers for messages coming from the server (Network::Receive) or from the client (Network::ClientReceive), which store information from network messages to be used later in other modules (like the AI).

Network::Send is the message sender class. It encapsulates network messages into simple, easy-to-use functions to be used outside of network subsystem.

Any descendant ServerType class may customize message handlers and parser helpers, but they should refrain from that other than for servertype-specific features and debugging.

The Network::Receive::ServerTypeX and Network::Send::ServerTypeX class

These are serverType description classes. They describe network message identifiers and structures for different servers. But they shouldn't do much else.

Note that none of these classes, from the connection manager to the serverType descriptions, should contain any AI code.

How it all works together

Main initialization code creates a connection manager instance and a message tokenizer instance:

$net = new Network::DirectConnection;

$incomingMessages = new Network::MessageTokenizer(\%recvpackets);

Connection manager creates a packet parser instance:

$packetParser = Network::Receive->create($wrapper, $serverType);

Main loop passes information from connection manager $net to message tokenizer $incomingMessages:

$incomingMessages->add($net->serverRecv);

Message tokenizer with data and a message handler $packetParser are passed to a packet parser $packetParser to process all available messages:

@packets = $packetParser->process($incomingMessages, $packetParser);

# compare with outgoing packets:
# @packets = $messageSender->process($outgoingClientMessages, $clientPacketHandler);

Packets are passed back to the connection manager, which passes them to XKore clients:

$net->clientSend($_) for @packets;

Meanwhile, the packet parser calls custom parsers, hooks and built-in handlers:

my $custom_parser = $self->can("parse_$handler_name")
if ($custom_parser) {
	$self->$custom_parser(\%args);
}

Plugins::callHook("packet_pre/$handler_name", \%args);

my $handler = $messageHandler->can($handler_name);
if ($handler) {
	$messageHandler->$handler(\%args);
}

Plugins::callHook("packet/$handler_name", \%args);

Handling multiple server types (message parser part)

Back in 2003, things were simple: you had iRO (International) and kRO (Korea), and that's it. But today there are more than 5 official RO servers (Brazil, Philippines, Europe, Russia, etc.) and hundreds of private servers. All those servers have a slightly different protocol. If would be very inefficient to create a different Kore version for each server protocol type, because although the protocol differ, they don't differ too much. That's why OpenKore has to be able to support all (or at least many) of the different RO server protocol types.

Thus we introduced the concept of a server type. Maybe you've seen the "serverType" option in servers.txt. The number specified in serverType specifies what kind of protocol that server uses. For example, at the moment (December 20 2006) we have:

Server type 0 for iRO, pRO, AnimaRO, and many other servers. Server type 8 for kRO. Server type 10 for vRO. ...etc... This is not a complete list of server types. We have 18 server types at the moment this page was written.

The Network::Receive and Network::Receive::ServerType0 classes contain code for parsing the server type 0 protocol. There is a subclass of Network::Receive for each server type. Each subclass is specialized in parsing the server protocol for that type. The class hierarchy looks like this:

When the connection manager connects to a server, it looks up (using the information in servers.txt) what server type is associated with that server, and instantiates the corresponding message parser class for that server.

Implementation details

If you look at the 'new' method in Network::Receive, then you see something like this:

$self{packet_list} = {
    '0069' => ['account_server_info', 'x2 a4 a4 a4 x30 C1 a*', [qw(sessionID accountID sessionID2 accountSex serverInfo)]],
    '006A' => ['login_error', 'C1', [qw(type)]],
    ...

$self{packet_list} is a hash which maps a message ID (sometimes also called packet switch) into an array with the following information:

The function which will parse this message. In the piece of code above, 'account_server_info' is the name of the function which will handle that message. How the message looks like, i.e. the message structure. These are funny strings like 'x2 v1', which are passed to Perl's unpack() function. If you don't know what they mean, read this and this. A list of variable names associated with the message structure. It specifies the variable names for the structure, in the same order as the unpack string. In subclasses, you can modify this hash at will. You can also override handler functions to do something else, depending on how you want it to behave for that server type.

The following arguments are passed to the handler functions:

The packet parser instance. The message variables as a hash reference, which we call the message arguments. In the above example, account_server_info would look like this:

sub account_server_info {
    my ($self, $args) = @_;
    # $args->{sessionID} can be accessed.
    # $args->{accountID} can be accessed.
    # ...etc...
}

The message arguments hash has two special items:

$args->{switch} This contains the current message's ID. $args->{RAW_MSG_SIZE} The size, in bytes, of the current raw message. $args->{RAW_MSG} The raw, complete message, including message ID. If the message structure cannot be described in an unpack string (in packet_list) then you may process the data manually using this variable. Note: RAW_MSG may contain more data than just the current message. It may also contain (parts of) the next message. As a rule of thumb, never process more than RAW_MSG_SIZE bytes.

Example 1: adding a new message handler

For example, let's say the RO server recently added a new packet which says "you are dancing", and that that message also tells you where exactly you are dancing and what kind of dance it is. Let's say the message looks like this:

Its message ID is "1234". It has three fields in the message: a 16-bit integer for the X coordinate, another 16-bit integer for the Y-coordinate, and a 1-byte field which says what kind of dance you are doing. Then you add this entry to the $self{packet_list} hash:

'1234' => ['dancing', 'v v C', qw(x y type)]

Here, the 'v v C' says that the message contains a 16-bit integer ('v'), followed by another 16-bit integer ('v'), followed by a 1-byte character ('C'). The 'qw(x y type)' part says that the first field should have the name 'x', the second field should have the name 'y', and the third field should have the name 'type'. Reference: https://perldoc.perl.org/functions/pack

And you also add the following message handler function to Network/Receive.pm:

sub dancing {
    my ($self, $args) = @_;
    message "I am dancing on position ($args->{x}, $args->{y})! My dance type is $args->{type}\n";
}

And that was it! Everything else is automatically handled by the framework.

Example 2: handling a different server type

Let's say the server with server type 12 has a slightly different "you are dancing" packet, in which the X and Y coordinate fields are swapped. That is, after the Y-coordinate comes the X-coordinate (instead of first X and then Y). You can modify the packet_list hash to update its message structure. So in the 'new' function of Network::Receive::ServerType12, you write this:

sub new {
    my ($class) = @_;
    my $self = $class->SUPER::new;

    # BEGIN: YOUR CODE
    $self->{packet_list}{1234}[2] = [qw(y x type)];
    # END: YOUR CODE 

    return $self;
}

If your Perl is not good enough: recall that $self->{packet_list}{1234} looked like this:

['dancing', 'v v C', qw(x y type)]

You wanted to modify its variable names list, which has index 2 in this array. That's why you wrote

$self->{packet_list}{1234}[2] = [qw(y x type)];

And now, $self->{packet_list}{1234} looks like this:

['dancing', 'v v C', qw(y x type)]

Handling multiple server types (message sender part)

Handling multiple server types in the message sender part is almost the same as the message parser part. The class hierarchy looks like this:

The difference is that the server type 0 handling code is in the class Network::Send::ServerType0. All other Network::Send::ServerTypeX subclasses inherit from Network::Send::ServerType0.

Interestingly, most of the difference between server protocol types is in the sending part! The messages that we receive from the server don't differ a lot between different server types, but the messages we have to send to the server differ a lot more.

Using the message sender

Like the message parser object, the message sender object is also created by the connection manager. That object is stored in the global variable $messageSender. To send a message to the server, write:

$messageSender->sendFoo();

Each message sender function has the prefix 'send'. See Network::Send::ServerType0 for a list of possible sender functions. For example, if you want to send the "public chat" message to the RO server, write:

$messageSender->sendChat("hello there");

Compatibility notes

The object-oriented message sender architecture (as described) on this page was introduced on December 20 2006. To send a message to the server in earlier OpenKore versions, you had to write one of these:

OpenKore 1.6 and 1.9.0-1.9.2 syntax

sendFoo(\$remote_socket, args);

OpenKore 1.9.0-1.9.2 syntax

sendFoo($net, args);
$net->sendFoo(args);

These won't work anymore. Instead, write:

$messageSender->sendFoo(args);

Or, if your plugin must have 1.6 compatibility write this:

if (defined $Globals::messageSender) {
    $Globals::messageSender->sendFoo(args);
} else {
    sendFoo(\$remote_socket, args);
}

Hooks

The message sender class provides some hooks which allows plugins to handle messages.

"packet_pre/$HANDLER_NAME"

Here, $HANDLER_NAME is the name of the handler function, as specified in the packet_list hash. For example, "packet_pre/account_server_info".

This hook is called just before the handler function is called. But this hook is only called if there is a handler function for the current packet. The argument given to this hook is an array containing the message fields (the message arguments).

"packet/$HANDLER_NAME"

This hook is called after the handler function is called, or when there is no handler function for the current message. Its argument is the message arguments.

Appendix A: introduction to the Ragnarok Online protocol

The Ragnarok Online protocol uses TCP as its transport protocol. Every message* that the RO server sends to its clients has the following format:

packet switch (2 bytes) + message content (variable length)

A message consists of at least 1 field: the message identifier (also known as the packet switch, but I think "message identifier" is easier to understand). This message identifier tells the client what kind of message this is. How the message is to be interpreted depends on this message identifier.

There are two kinds of messages:

Static length messages

These messages are always of the same length. Examples of such messages are the "someone has sent an emoticon" message and the "a monster has appeared" message.

Variable length messages

The length of these messages depend on their contents. Because their lengths vary, they have a special message length field which tells the client exactly how long the message is. This length is the length of the entire message, including message identifier. The "you have sent a public chat message" message is an example of a variable length message. Finally, we have the message arguments. The exact contents of the arguments depends on the message. For example, the "someone has sent an emoticon" message has the following information in its message arguments:

The ID of the actor who sent the emoticon. What kind of emoticon it was.

(*) There is one exception to the rule. If the client is in-game, and the user instructs the client to switch character, then the client will disconnect from the map server and connect to the character server. The first message that we receive, in this case, is the account ID, which is exactly 4 bytes. It is not a "normal" RO message in that it has no message ID - it's just a serialized integer.

Appendix B: recvpackets.txt and handling message lengths

When we receive data through a socket from the RO server, we cannot assume that we receive exactly 1 complete message every time we read from the socket. We may receive a part of a message, or we may receive two messages, or we may receive a complete message and an incomplete part of the next message. This is why we must buffer data received from the RO server. Whenever we've determined that we've received at least one complete message, we'll process that message and remove it from the buffer. Then we keep waiting until we know we have another complete message, and so forth.

But how do we know whether a message is complete? To know that we have to know every message's exact length. That's what recvpackets.txt is for: it specifies which messages have what length. For example, recvpackets.txt has this line:

00C0 3

This means that the message with identifier "00C0" is a static length message, and has length 3. But sometimes you also see a line like this:

00D4 0
00D4 -1

The 0/-1 means variable length, so in this case it means message 00D4 is a variable length message. As mentioned in appendix A, variable length messages have a message length field which tell us how long that message is.

Appendix C: obfuscation of outgoing messages

RO has made several attempts to prevent third party clients from (correctly) accessing the server. The most important attempts involve the obfuscation of outgoing messages. That is: messages that are to be sent from an RO client to the RO server are first obfuscated using some algorithm. This appendix describes a few obfuscation techniques.

Padded packets

This is not used anymore Some RO servers, such as euRO (Europe), iRO (International) and rRO (Russia) use so-called padded packets. The RO client will insert what seems to be garbage data into parts of certain message. In reality, this "garbage" is the result of of a complex hashing algorithm. Furthermore, the size of the garbage data, and the algorithm that is used, varies every time a sync is received from the RO server. Thus, a packet may have different sizes during different times.

We refer to messages that contain such data, as "padded packets". The padded packets emulator subsystem in OpenKore is responsible for generating correct padded packets.

Padded packets only affect packets that are sent from the client to the server, not packets received from server to client. Furthermore, not all packets are padded - only some are, usually the "sit", "stand", "attack" and "use skill" packets.

See the file "src/auto/XSTools/PaddedPackets/README.TXT" in the OpenKore source code for more information.

Encrypted message IDs

In this technique, the message ID of an outgoing message (i.e. the first 2 bytes) might be encrypted. This is only applicable to messages that are sent to the map server - account server and character server messages are unaffected. Messages are unencrypted, until the map server, at some point, sends the encryption key. This encryption key is valid as long as the connection to the map server is alive. Once the client disconnects from the map server, the encryption key is invalid and should not be used.

See the file "src/Network/Send.pm" in the OpenKore source code for the exact algorithm. Look for function encryptMessageID().

Original article

http://web.archive.org/web/20090305035837/http://www.openkore.com/wiki/index.php/Network_subsystem