RE: Xap and serial

["Ian Davidson" <ian@xxxxxxxxxxxxxxxx>]

Hi Kevin

Thanks for the reply. After I had sent the email I had a think myself
and decided a type of filter app would probably be the best solution.

I'm very much out of depth here but this is my thoughts. The devices I
wish to use would work similar to netiom although dedicate to a task
with both i/p and o/p. I us some of the netiom outputs to control a
couple of flood lights. Looking at the Xap messages when homeseer sets
these lights on or off an Xap bsc message is sent targeted at the output
of the netiom. In my case I am only interested in these messages. I know
I could (well if I could program for the PC) write an app to filter for
specific devices but my idea would be to filter for the target address
line. If it was filled in then allow the message across the bridge if
not then disregard it. This way the bridge should not need mapping all
the time but should reduce traffic levels to acceptable via a simple
serial device. Interested on your thoughts to this.

Only one bridge would be required and I presume I would need to use a
different port number for the output of the bridge. Then tell any of my
devices the new port which would mean they will only see targeted Xap
messages.

The next thing would be to write the bridge application. I don't really
write PC based apps although I have recently experimented with VB.net
express and managed a couple of small apps. I have just downloaded some
of the Xap framework stuff but I have only found C based source files.
Do you know if there are any VB based source files I could look at and
then try to write a small Xap app from it?


Thanks

Ian D


-----Original Message-----
From: xap_automation@xxxxxxx
[mailto:xap_automation@xxxxxxx] On
Behalf Of Kevin Hawkins
Sent: 18 February 2007 01:30
To: Ian
Subject: Re: [xap_automation] Xap and serial

Hi Ian,

Bear in mind that the application you are using bridges all xAP traffic
to serial. In any fully bridged network transport you are restricted by
the maximum speed of the slowest network (quart into a pint pot
scenario). There are some things you could employ to improve this eg
compression etc and obviously any 'bridging' application should be
implemented with adequate buffers that can buffer transfers between high

and low speed networks. The same would be true of a simple115200 to 2400

baud inline buffer. Thus any burst would be paced appropriately onto the

lower speed network. A Tibbo therefore might not be the most suitable
device for this application. When I transfer Etherene originated
commands to serial C-Bus at 4800 baud the C-Bus serial interface buffer
space is only some 20+ characters so lots of buffering here. HomeVision
has only single command buffers - sometimes 2 bytes.

That aside I think it's important to first decide... is all that traffic

necessary in the first place ? Why have you got so many devices xAP
enabled and reporting status ( I assume) in HomeSeer - are they being
monitored elsewhere on xAP ? Then secondly why are you onwardly
transferring all this info to the serial port - are there devices on the

serial port that also always need this information.and if not why do you

send it ? There are various filters that can be employed to reduce the
traffic that is passed and it may be that you need to configure a
selective bridge (router) to filter unwanted information. You will know
which devices on the serial side are listening to which devices so you
only pass only the information that those devices need (filter on
source) - this can be further filtered by the class as you probably are
only interested in a particular schema. Additionally though you should
always pass any packets specifically targeted at devices sitting on the
serial link. This will reduce a whole lot of traffic - maybe 90% or
more. I tried this in an application with Crestron processors where the
Crestron processor registers with a 'router' the devices it is
interested in and the router then only forwards info from these devices.

Given your stats and assuming my 90% is a reasonable figure then you
have achieved a serial baud saturation of 15138 baud - very comfortable.

In fact given your stats you only need to see a 25% reduction but more
is obviously better.

If you do need really verbose realtime interaction between many devices
then xAP is probably not the most efficient protocol and there are other

bit stuffing protocols like 'SNAP' etc that truly compact this data. In
which case why not utilise SNAP or similar on the serial leg and bridge
this protocol to xAP with ample buffering. Implement a xAP to SNAP
conduit . What you gain in message size you will generally sacrifice in
message scope/expandability and certainly readability though.

You could (should you wish) also build on top of xAP a pacing layer
between devices - something that exchanges xAP packets between two
devices with many extra checks in place including extra error checking
,acknowledges, retries continuation packets etc - I guess a sort of
halfway house between XONXOFF and TCP. Indeed for some devices you
could even implement TCP tunnels but you have to be careful not to
compromise the methodologies of xAP. If you want 'guaranteed' delivery
use a custom schema that acknowledges receipt. These sort of features
may be formalised as part of a later xAP protocol update , maybe using
additional parameters within the header. We will not change or break the

existing protocol (it has remained solid for years now) but we can add
extra layers on top of it for those that want these type of features.
Being able to do such things is one of the great strengths of xAP and
you can do this now yourself anyway.

Re the Netiom - this is actually one of the fastest xAP devices around
and in an experiment I set up an output on a Netiom was monitored and if

it changed it was reversed by a PC application (xAP Mapper) and I was
achieving a throughput of over 6000 messages per minute - this was a
self pacing test - that is some going - and the limitation might still
have been the PC application rather than the Netiom . I have never known

the Netiom miss messages but I suppose if it was receiving very fast
Ethernet speed requests destined to output lots of serial data at a low
speed then if there is insufficient buffering something has to give.
Either the receiver (Netiom) has to throttle back the sender (XONXOFF
style) or data will inevitably be lost . Note it is still not missing
incoming data but it is unable to action its content fast enough. On the

receive side as Netiom is always transferring from low speed to higher
speed layers nothing can ever be lost. The other key aspect of the
Netiom is that it parses incoming xAP data inline / on the fly (unlike
other xAP libraries that reform xAP messages into structures for later
analysis), this is a different approach that embedded designers need to
implement - and indeed is something I am still learning with the
embedded xAP gateways I have coded. It's all to do with how long it
takes you to finish processing an incoming message and this can be quite

fast (unless a lower speed transport is involved). On a Netiom (barring
the serilal output port) it's pretty instant.

It is true that it certainly helps if xAP applications that are sending
updates on a lot of endpoints stagger these over a period, particularly
in BSC. Indeed on the Netiom you can configure it to step through each
endpoint one at a time over several minutes rather than burst them all
at once. When responding to a wildcarded xAPBSC.query command addressed
to every endpoint then it is important that all of these respond fairly
instantly as this provides a limited form of discovery. Off the top of
my head I suggestion in another thread at least 2ms per response which
gives a flood rate of 500 messages per second - but it may be that a
recommendation of longer than this is more desireable . Maybe 20ms
giving 50 messages per second ?

Regardless of any such a recommendation I think the key to your solution

is in choosing appropriate hardware to implement bridging. You need to
consider your peak xAP data arrival burst rates and the level of
filtering you can apply to reduce (filter) the number of messages to
transfer to the lower speed transport - and then ensure adequate buffers

are available or data will be (has to be ) lost. As such PC based
applications make excellent platforms for such applications. Otherwise
you need to code a 'bit mapped/low level protocol' to xAP conduit.

Kevin

Ian Davidson wrote:
>
> Hi
>
> It's been quite a while since I did any development work with Xap and
> when I did it was only minimal. However I decide I would try to ease
> myself back into it as I get a few minutes.
>
> I mainly work with PIC's but did want an Ethernet device. Rather than
> trying to put Ethernet directly into the Pic (I know it's possibly but

> not by me!) I decide to use a Tibbo device to act as the Ethernet to
> serial gateway and just let the Pic talk serial.
>
> The Tibbo (www.tibbo.com) is quite powerful, but for the first tests I

> simply programmed it for UDP and to forward all traffic on port 3639
> to serial with a serial rate of 115200 Baud.
>
> In a terminal program I monitored the output and saw my Xap traffic
> happily flowing away. After a while and pausing the display it became
> apparent that some messages were broken. I then used the Xap viewer
> side by side with the terminal program and could see some missed
> messages. On closer inspection everything was fine until the Xap
> homeseer plug-in sent a large amount of Xap messages. Looking more
> closely at this homeseer actually sent out 69 Xap messages in 0.84
> seconds. The average message length was 222 bytes which equates to
> 15318 bytes. To send this serially would require a start and stop bit
> as well as the eight data bits which equates to 153180 bits in 0.84
> seconds or 182357 Baud.
>
> This is why I was missing and getting some corrupt messages. I know
> originally Xap was aimed at a multitude of devices but this rate of
> output clearly rules out almost all serial devices. Even if the Pic
> did have a direct Ethernet interface it would be pushed to cope with
> this rate of data. I have the Xap Netiom which is great and I know
> uses a Pic with direct Ethernet but I'm sure it would not be able to
> cope with the high data rate at times. Although I have not had an
> issue with my netiom this is probably due to the commands to it not
> being sent during one of these data storms.
>
> I'm really just looking for comments on this I would like to use Xap
> on Pic based devices but I would also like to guarantee it didn't miss

> any messages and as it stands I can not. I'm looking to control some
> devices and having used Xap, which is great and still growing would
> have seemed ideal but at the moment it seems as though a lighter
> protocol would probably be more beneficial. Can anyone think of a
> solution, is it possible to limit the speed of Xap packets onto the
> network or would this start to limit and slow other Xap PC based apps.
>
> Thanks
>
> Ian
>
>