A note on HooVorNet Memory Performance.

A feature of my HoovorNet is it’s minimal use of dynamic memory. Take a look at my general architecture for HooVorNet below. The input is 256 x 256 x 3 and the output is 16 x 16 x 768. If you carry through those multiplications you will find that 256 x 256 x 3 = 16 x 16 x 768, which means the output uses the same memory as the input, and if you look closely then you will see that the largest use of memory is the output of the first feature generator which is only 150% of the input memory. And HooVorNet doesn’t employ any sort of bottlenecks or hidden memory expansions. This max pipeline memory for HooVorNet is 75% the max memory used by MobileNetV2. The parameter memory is quite small at around 4700% of the size of the input image, but not as small as some of the implementations of Mobilenetv2. This means that the static parameter memory of my HooVorNet is larger, but the dynamic pipeline memory use can be smaller, which I believe means that it is possible that more pipelines of HooVorNet could be simultaneously processed in parallel on hardware then could be with MobileNetV2 pipelines.

HooVorNet: The Hints of Ozone Vortal Network

I’ve decided to call my segmentation network HooVorNet, and I’ll share some its key unique features. HooVorNet employs feature generator blocks that split filters among several calculation paths and then recombine with concatenation. HooVorNet’s feature generator blocks are distinct from other feature concentrators that use a similar method such as Google’s Inception Module because HooVorNet uses shallow and deep auto-encoders in each of the split paths, and incorporates a skip connection by concatenation.

Using the HooVorNet Feature Generator block, the network simultaneously gains the benefits of normal convolutional feature examination and the incredibly deep inference of chained auto-encoders. This block encodes shallow swept local features (1×1,3×3, and 5×5) and deep strided features (16×16).

The network employs special and undisclosed deep feature generator blocks just ahead of the output that are a more advanced version of the Feature Generator Blocks that provide the same benefits while keeping parameter and FLOP counts of the network at a minimum. For this particular embodiment of HooVorNet, the full model has only 9 million parameters and requires 4 billion floating point operations.

Segmentation Update

I’ve created an auto-encoder based image processing model which accepts a 256×256 pixel rgb image and outputs a 256×256 pixel rgb image. One of the uses of this model is single class segmentation, and I’ve successfully trained the model to identify people and generate a black and white output image where all of the pixels of the input image that the model believes are part of a person are white and all of the pixels that the model does not believe are part of a person are black.

The success of my model in the performance of this segmentation task proves that the model identifies features and makes decisions based upon them. This feature extraction and processing is carried out in the model’s encoder which utilizes a special combination of feature extraction techniques. The model’s decoder portion is much simpler, and really just transforms the encoder output into the desired 256x256x3 image output. This decoder section can be replaced with other submodels to use the encoder output for such tasks as classification, multi-class segmentation, style-transfer, and others.

For the single class segmentation which I’ve trained it for, the model utilizes 9 million total trainable parameters in 4 billion FLOPs (floating point operations), making it great for mobile applications. As a comparison, a Vortal photo filtering of a high resolution 3600x4800x3 image could utilize trillions of FLOPs to process the image. Now that I’ve made this new model, I could use this for developing a feature for Vortal which permits a user to decide not to apply the Vortal transformation to automatically detected people that are in the photo, or alternatively, only to the people.

Segmenter Progress

I’ve continued to improve my segmenter. Here in the above, I’ve used it to detect people in images that aren’t part of the training set, and I’ve used the output to then redact the people from the images. These images are free for use from pexels.com. My segmenter is best at “chest and up” pictures of people, but it’s been trained on various group photos and various poses. This segmenter does not use query images, and instead it operates in line. A 256 x 256 x 3 channel zero (black) padded image is input, and the output is a 256 x 256 x 3 image where the pixels that are “part of people” are white, and the pixels that are “not people” are black. That output image can then be used with lots of different types of post processing to do things such as I’ve shown above where the people are redacted. The segmenter model consists of two sub models: An encoder with 8.37 million parameters and a decoder with 0.63 million parameters.

Here’s a little scene from something I’ve been working on. A survival story set in a world I’ve been building.

A blast of force and fire woke us, likely from Wryan who’d had the second watch that night. I leapt out of bed, just in time to avoid the swipe of a bony scythe the size of a tree which tore through the tent canvas and my bedding before lifting the whole tent away, ripping away its tiedowns. I reached my senses out reflexively, so I sensed another scythe slicing in from the right and ducked. Marris was cleaved in half by the brutal swipe as she struggled up. I took off at a sprint away from the general direction the scythes seemed to be coming from, and I called to the dirt, air, and wood around me. The materials answered my call and dissolved into dust in whip like tendrils from the earth and trees that I rushed past. This matter chased after me in a cloud much bigger than my footsteps could ever have raised, ready to be formed to my purposes. I sensed I’d gained some distance from the monster, and turned.

The thing was all hard bone and magic. Numerous wide spade edged legs surrounded its shining core like a shield wall, and it held at the ready three long segmented limbs ending in the familiar deadly scythes. The spade legs were slicing forward in sequence as the monster built speed toward Wryan who floated among the trees with four swirls of matter rotating in the air in front of him. I rose in the air myself as two of Wryan’s matter swirls tightened their spins, flashed bright, and streaked towards the monster. The other magister immediately floated back to maintain distance, as one of his bolts shot straight and fast towards the monster’s bright core and the other followed an arcing path over the wall of its legs. I gathered more matter and shaped it into one great brown swirl above me. Wryan’s straight shot splashed into the creature’s foremost spade legs and the force and fire did little but rock the legs back and leave a blackened soot on the hard bone. Meanwhile the creature met the arcing shot with blurring swipes of its scythe arms, breaking up the force and flame before it could reach the core.

Wryan had paused in the air to draw some small dark objects from the pockets of his robes which he threw into his two matter swirls as he called up more matter from around him to start forming two more. The creature sliced on towards him. Wryan was adding some of his antimagic slugs to his next shots, small chunks of magic destroying material encased in an inert shell, one of the most effective tools in a magister’s arsenal. My own antimagics were lost in the tent, but I could help to give Wyran some time. I willed my large matter swirl to contract and ignite, and sent it forward with high speed and force. My large shot hit the thing’s spade legs harder than Wryan’s shot had and it had to pause to regain its balance. It didn’t seem to have been injured and after one of its scythe arms poised in my direction for an instant, an acknowledgement of my status as a threat, perhaps, it carried on towards Wryan.

There is a tendency, in a heated battle, for magisters to direct all of their expanded sense and attention toward one narrow area, in this case for Wryan and I, toward a vicious boney horror that had just slaughtered our friend. We train to reduce this sense tunneling, to set aside some small part of our attention to watch for certain threat indicators around us, but to achieve that successfully generally requires some preparation and focus. When a magister has time to prepare for battle, there is time to build enhanced attention structures and myriad weapon preparations which have historically made a unit of magisters able to challenge the Gods themselves. But there simply wasn’t time to prepare all of that this morning, and the creature might well have stealth abilities which had permitted it to sneak up on camp without giving Wryan time to alert us.

Regardless of the explanation, I was only alerted to the ambush when my call for more matter to prepare another shot met hateful, unyielding bone. I shot forward through the sparse and mostly barren trees and narrowly dodged the swipe of a scythe from a second monster. My heart felt like it would explode from my chest from the exertions of will, and I had to consciously force my body to take a breath. I reached sense toward Wryan’s position and encountered a wave of shock and pain from his dying mind. A scythe from a third beast had pierced his chest and his swirls of matter dispersed into brown disk clouds as the will which had called them disappeared. The third beast swiped its corpse laden scythe past a tree to dislodge its gruesome burden, and then all three monsters were converging towards me.

With no allies left and no weapon but my will, I flew. These hills were filled with dangers beyond the immediate threat of the scythe-beasts, and it is unwise to stay airborne here long even if it weren’t an exhausting means of transport. I flew up and over a low ridge and circled around a hill and down into a shear sided valley. It was probably only a mile from the camp but, it’d be a much longer hike, even for the shovel-legged scythe-beasts. I came down by a creek, gasping from the exertion of battle and flight. My mouth was dry and I scooped up some of the cool water without stopping to check if it was tainted. The water tasted sweet. I crawled up against a stone which blocked the early sun and put my back against it, crossing my legs in front of me. I closed my eyes and waited for my heart to settle. Not dwelling on the morning events yet. Not recalling the last mildly annoyed look on Marris’ face, nor the last echo from Wryan’s dying mind.