Hard G Soft G

Me:  "It's GIF not JIF"

Also me: "It's Rejex not ReGex"

Quarantine - Code-id 19

For two months now, we've been social distancing. Luckily I have a job where I can work from home. This saves me an hour at least per day in commute time. Except, now I have to home school too, but that's fun. It just takes time.

In order to be more efficient, I try to automate everything. Everything I write that is not directly doing work for a work project, is doing something to make me more efficient and also limit the amount of grunt work I would have to do.

Recent Problems

Dev content is out of sync! 

Solution:  Write a dbsync tool. Luckily the db we deal with mostly has no identity insert auto incremented integer id fields. This makes it easier. The thing that makes it harder is that the dbs for which I wrote this tool have millions of rows, are frequently updated, and have blobs that are multiple megabytes in size. This one I haven't made public.  But it will take a config of two connection strings, one is the source, one is the destination, then it will also have table definitions along with a timestamp field to be able to tell which rows have been updated since the last time the tool ran. When it reads all of the rows from the source table, it will update the time stamp so it gets much less data the next time. I just run these occasionally for all of the different databases that I have to manage. Having up to date content for development is a headache reducer.

Configuration is complex!!

Sitecore uses about a hundred configuration files. We have to keep track of different settings across multiple environments, like development content management and a separate content delivery, then the same thing for UAT, and the a more complex configuration set for production, with a CM and two CD servers typically. I solved this during a build by having a build process take all of the base Sitecore configs, then copy over application specific configuration, then finally copy over environment specific configuration. Then a separate process deletes files where there's also a .deleted file along site of it, like something.config and something.config.deleted.  Then a separate processes merging updates to Web.config in the root. This is pretty useful.

The main problem with this is that if we inherit a site, or want to convert an older site that didn't use this configuration process. So I made a config split tool! So you have a folder with all of the Sitecore configuration files, then each environment's configuration files in other folders, and run cfgsplit with the proper parameters. It will remove all the files that are the same across all of the folders. Then you run another pass at it with slightly different parameters, and it will move all configs that are the same from the environment configuration files into a separate application specific configuration folder, where all of those configuration files are the same. After that you're pretty much done.

The other aspect of this is the Web.config. There's a similar process with this except it works on XML, not folder structures. The idea is the same though. However, some XML nodes are extremely tricky to figure out a unique identifier for. Like, <x><y><a id="unique" /> <b x="y" /></y></x>   where the b node is the same across many different instances of y, and where y never has any attributes but there are multiple in succession. You might think using indexes might be a good idea. But, we're comparing whether these nodes exist in the other file, and having them be identified by index would not match the other files, so I had to match by content.

For a node like x or y, I would grab a hash of the children's contents. This was a good unique identifier. For a node like b, I had to grab a hash of the siblings. If there are two identical y nodes under this specific x, my code will fail. But you can argue when that would be useful, having two identical nodes as siblings in XML. I won't share this code either. It is nearly complete.

A File is too big to copy over RDP!

This one was a fun one and only took me an hour. I was heavily inspired from my memories of downloading "warez" in college, where you'd get a RAR file as a bunch of files, instead of one large one. This code I can share, it is here. I simply call it parts. It just does that, takes a file and splits it into chunks whose size is passed in via a parameter. There's no check on that size, now that I think about it, but I can check to see how many chunks would result and maybe throw an error if that's over an arbitrary limit, like 100 or something. But if you want to create 2 billion files, go ahead.

Happy coding!

Only Two Things Are Certain In Life

Death and taxes, and not accounting for everything at the beginning.

Adventures in Woodworking

For a while I've wanted to dive into the world of woodworking. As with my foray into computer science 22 years ago when I started college, I know nothing about it going into it, only that I really want to learn it. So if it's anything like my computer science adventures, I'll break a lot of stuff.

I have a lot of ideas and needs for things that can be built from wood, as well as just the desire to learn how to do it, and have things that I've created. I think this is a little like my desire to learn guitar decades ago, and recording myself and getting better and better.

I was given a generous extra bit of compensation from my generous employer, and I've been paying off my debt recently down to $0 (except for mortgage and car), so with the extra money I saw the opportunity to just go ahead and dive right into my new hobby as a woodworker. I've bought the basic necessities, signed up for a 6 week course, and I'll probably just be doing that during nights and weekends. Although there is the quiet hours I have to worry about at night :)

I'm looking forward to building stuff. I'll make posts in the future with thoughts and experiments, challenges, etc.

New Car!

I've actually had my new car for 5 weeks today. It's the coolest car I've owned, by far. It's not the fastest car I've owned (may be slowest). It's the smallest I've owned (by 0.2 inches). I thought the 2002 Monte Carlo SS in Medium Green Pearl was the sexiest car I owned and would always be, but I think this one might top it.  Plus it gets insane gas mileage.

I've found if I drive normally, I can get decent gas mileage like 45 mpg. But I wanted to max it without modifying it in any way, so that just means I'd have to modify how I drive.  Basically, don't gun it. If I'm coming up on a red light, let go of the gas and try to EV my way there. Or coming up on a downhill stretch, just get off the gas and try to get to the top of the hill on momentum. It's a lot of momentum usage and trying to keep it in EV mode, and trying to charge the battery so I can use EV mode more!  I've been up to 56.2 mpg is my highest average after a decent chunk of miles put on the tank.

I've filled up twice.

So, for specs and all that... It's a 2019 Hyundai Ioniq SEL.  The SEL trim means I get safety features like the lane keep assist and cross traffic monitoring when I'm trying to back out of a spot, stuff like that. The smart cruise control is amazing.  It's a very comfortable, quiet drive. If I had one complaint it would be the cloth in the trunk and how it wraps close to the seat. It's not a big deal. Best car ever. Super sexy, hella gas economy. White with black interior, which is exactly what I wanted :)

Pictures!

Last tank I put on 543 miles on it before filling it up, put in 10.2 gallons in it, so averaged around 53 miles per gallon. It's kind of mind blowing.

Budget App

I'm a software developer, which, if it's not your first time here, or you know me in real life, then you know this. If I have a problem which can be solved with software, I write software. I was having a hard time keeping track of my money, and like, being able to project when hard times will not be hard. So I made a thing. I named it "ABIDE" due to the 5 things it tracks, namely, in correct order, "adjustments", "balances", "incomes", "debts", and "expenses". It's a very simple app, you enter in the data, and then it computes out as far out as you want to go. I've gone as far into the future as my projected retirement year, 2045.

Currently it's not open sourced, but it's on github and I'll just have to flip the private repo to public.

The premise is easy. Enter data, compute data. Occasionally enter in updated amounts since you can't necessarily track every single dollar in your life. Here are a bunch of screenshots that show it in action.

Adding and updating balances, your starting amounts for savings / checking accounts

Balances

Modifying expenses

Modifying expenses

The Calendar

Calendar

Reconcile, which is to update the calendar with actual amounts, without actually modifying your past numbers. Although you can still do that.

Reconcile

And the month summary will tell you the starting and ending amounts for each of your balances.

Month Balances

The project uses docker, and a MongoDB container, so your data is stored in the container, and containers can get wiped. So it's a good idea to grab a backup occasionally if you have changed the data! With the data transfer, you can import and export data.

Data Transfer

 

I will have more later, but for now, it's helping me a lot!  It's written in Go and Angular 7 as well as using MongoDB. 3 containers are spun up with docker-compose. Both the Go service and the website are multi-stage builds. It's neat. Code to come! Here's the docker compose file.

docker compose

It's not the best Angular. I was getting really lazy with it :P Anyway, check back soon, or hit me up on twitter or something (above) if you want to use it, I can send you the latest.

New Phone and Service

From my previous post on tidying, one of the ways I was going to "tidy my finances" was to switch phone providers.

My coworker has been on Google Fi for a few years, and he has always talked about how he's used so little data, and his monthly payment was in the 30-50 dollar range every month. While jealous of his payment, I always came back with the fact that I get unlimited data. Unlimited Data!! For less than $120 a month. 120 dollars!

The company advertised it as you get unlimited data and calling for $70 a month though. However, I was on a program where you can upgrade to a phone any time you want, up to three times a year. The phone leasing and various taxes and a phone protection plan apparently cost me another $50 a month. However, I don't remember ever asking for the phone protection. I leave the old company a little bit mad at them because of that, a fact that I only discovered when I was switching, and the fact that I couldn't turn my phone in and kill me lease, they would charge me for the remainder of the lease? WTF?!?  I still have the phone as the store said the main HQ will bill me later. At that point I plan on protesting it.

I'm now 10 days in with Google Fi and a new shiny Google Pixel 3 XL. Over those ten days I've used 150 MB of data. Over that same time I've used 15 GB over WiFi. I always thought I'd be crippled without an unlimited data plan, or 6 GB or whatever, but it's actually pretty easy.

Data Saver

Google phones offer a data saver. I thought it was new to my new phone because I never checked, never needed it. But it's there, and you only care if you're not on an unlimited data plan that is over priced. The data saver is an amazing piece of software! It will turn off background data for all apps, except for two by default, and for any that you choose to allow access to background data (none on my phone ...).  My data usage is at an all time low.

To get the data so low, though, you really need to make some sacrafices. No web browsing. No YouTube. No streaming music. These can be alleviated by downloading things over WiFi. I've downloaded a few albums. I download some podcasts before I leave for work or for the office. I have plenty of material to keep me entertained for many drives back and forth to work.

It will get tricky once I need to go somewhere I'm not familiar with, and use the Google Maps app. You can download your area to maps instead of streaming that data, but traffic needs to be real time!  But this isn't a data elimination, just a diet. I might go up to 1-2 GB, and that's fine, as my bill will still be less than $50. $68 less than what I used to pay.

Phone

For the phone, like I said I was leasing phones with the other company. To get rid of leases, I bought the phone on my credit card. $800 total! Yes, it's steep, but I will pay it off quickly. The way Google Fi works, it'll only ever charge you for 6 GB of data if you go over. With $20 base and $60 for data at the most expensive payment (add some taxes) for the Fi a month, I'll still pay $20+ less than the old plan. If I did that and used maximum data every month, the phone and plan switch would pay for itself in 40 months. If I use 1-2 GB a month, figuring my monthly payment is $68 or so less than my old plan, the switch pays for itself in 12 months. If I shut off data completely, I'm $93 cheaper, the switch pays for itself in 10 months.  Figure I'll own this phone for a few years, at which point I can trade it in and get the next faster phone cheaper. This is big time savings and I was stupid for not doing it a while ago.

As long as Google Fi exists, I won't be on a major carrier. It's plain dumb.

Tidying

Over the past few weeks, I've been on a tear with cleaning up my place. I watched the Marie Kondo Netflix series, and I was kind of inspired by it. I decided to try it on my clothes. So I went through everything, got rid of stuff that didnt' "spark joy" (more on this later), and folded everything into tiny little squares!  At the end of my process, I had 2 empty drawers, where they were difficult to shut before they had so much in them! And the folding everything made everything just fit much better.

Inspired by the space saving produced by my clothes experiment, I decided to do the rest of my rooms in the upstairs part of my house. The collection of stuff elsewhere up there wasn't so drastic, so I moved on to the downstairs.

The living room and dining room are particularly cluttered. I have bikes, toys for Genevieve, loads of board games since that's what we like to play, tons of books, various things, video games, pictures, a shovel in case it snows, vacuum, the lone closet on the first floor is packed to the point of overflowing. During my initial flame of inspiration to declutter, I didn't make much progress on the first floor because there was just nowhere to put anything. Because my basement...

My basement is a nightmare!  When I moved back in to my house 4 years ago, everything I brought with me just made it into a pile down there. As I needed things I would bring them up. When I didn't need things, or had loads of boxes from online purchases, they made their way downstairs. I was always afraid to throw things out because I'm not sure of the impact on the environment, if the trash truck would even take them, and probably because I'd always forget.

So yesterday, March 2, I worked all day to clear it up. I threw out a lot of stuff. I found my PS3 during the day and noticed that all the games I have are in mint condition, and they were all in their boxes! I went and sold them, trying to sell my PS3 as well, but I was missing a wire. I ordered it so I should be able to sell it Wednesday. (Hopefully the sale nets me more than $10 :)

I ordered Sterilite boxes to store stuff in, as well as a metal rack. I already had a 4 shelf rack down there which quickly became consumed by bedding and Christmas and Easter decorations. I got a 5 shelf one the other day. The Sterilite boxes I got were four (for lack of a better term) big ones, and 6 (again) small ones. I'm sure they have liter measurements though.  The whole process is nearly complete, and when I'm done here, I'm going to package the rest of the loose ends down there, throw away the remainder of the boxes, figure out what I can sell, and then it will be done!  And I will have room for the rest of the stuff on the first floor, to be packaged up or determined to be sold or trashed. It's a liberating experience. I feel I'm reclaiming my house!

Financial Tidying

At the same time... I'm doing what can only be described as financial tidying. In the same way that I'm making room in my basement for stuff upstairs, I'm making room in my monthly expenses for a new purchase that I'll be making. I mentioned this earlier in my Hybrid Car Shopping post. Some things I'm doing there are

  1. Switching my phone provider to Google Fi. The initial cost is the phone which was $800 (I plan on paying it off quickly), but the monthly cost will be under my control, between as low as $20 and as high as $80. Before taxes I think. However, my current phone bill is $120. The variable pricing with Google Fi is how much data you use, at $10 per GB. So if I can try to limit that, and I typically only use 3GB a month WITHOUT moderation, my bill will probably be $50-60 cheaper, and it'll have paid for the new phone in a year.
  2. Paying off credit cards. These I didn't really used to count as monthly payments for some reason. That was a mistake! I was making a little more than the minimum payment each month, which was something like $200 a month across 2 cards. The debt is not outrageous but if I can get rid of them quickly, I can gain back $200 a month right there! Easy!!
  3. Paying off my lawyer.  However, this is zero interest (so far!) and not of the highest priority. However at the rate I'm going it'll be paid off in a year. He's likely to do more work for me though, but that's ok, and welcomed since he provides a valuable service :)
  4. At some point I'm going to learn Google Cloud stuff and have my websites be functions in the future, and not need a virtual Linux server like this site is running on. I've already moved over all of my private Git repositories to Github since they offer unlimited private repos for free!  FREE!

Those are the things I can do right now. I have no interest in paying extra principle (see what I did there?) on my house since I was going to try to move but now I don't feel I need to. If I can get my house into tip-top shape and be happy living here. There are a few things that I want to do in that regard.

  1. New furniture. This includes a new couch and new mattress. These will likely not happen until I get my monthly payments down to a minimum, namely paying off the credit cards.
  2. New refridgerator. It's just old and the lights don't work, etc. However, this will be very low priority, as I might go in there and reorganize to start feeling better about it.
  3. Fix plumbing. I have a leak in my bathroom piping. This is high priority.
  4. Fix my garage door. This is medium priority.
  5. Nest thermostat. This could also help with my monthly bills.

In terms of the Marie Kondo method, and the "sparking joy" concept of it. It's definitely not only about joy but also function. I'd definitely not throw away a pair of sweatpants because they don't spark joy, but keep them around for the few times I actually feel like lounging around in them. It's hard to assess for me, the joy that is caused by material things, since I'm not a material girl, or even a girl for that matter. I do have a lot of stuff but that's just because I have interests. I have about 50 piano books. Old cameras. Video games and consoles (like an Atari 7800 and a Sega Genesis). Tons of guitar stuff. Computer stuff. My hobbies spark joy. However, the space these things take up while not organized and put away neatly sparked the opposite of joy. And that's what I'm addressing :D

On the financial side of things, credit card debt does not spark joy! :P  Debt in general can kiss my butt. Monthly payments do not spark joy. A new shiny car with sweet tech and high gas mileage though... In order for me to fit that in I had to "make room in the basement". The difference between the financial tidying and the house tidying, is that I am not making room in my house for new stuff. Just making room for breathing, and for joy.

I often think... did watching Marie Kondo light this fire? Or was it the timing of me like, coming to a point in my life where I feel cluttered and overwhelmed with loads of crap, a bit of debt (again, not ridiculous), wanting to introduce something new in my life?  Like a car? Or perhaps start dating?  Or is it all just Marie Kondo and her contagious joy?  I think it was Marie Kondo :)

Thanks for reading! It's a journey.

SCGen Update

With some client Sitecore work coming up, I've had to think about how to get rid of TDS for that specific client. I haven't had to do much Sitecore template work on that project at all, so i've been able to exclude TDS projects from the solution pretty much from the beginning.
 
However, the TDS code generation within that project created a Model.cs file that is nearly 53,000 lines. 2.4 MB!! It is pretty monumentally important that scgen can generate pretty much the same code, but without the enormous overhead of TDS. However, not the same code, as much of the code in that Model.cs is repeated, like full repeated "using" statements for every single type... (ugh).  I can probably get it down to half the size, or even better!
 
The TDS code generator was generating things that weren't covered by scgen, like field wrapper properties along with field wrapper value getters.  It was generating solr index attributes. Index field attributes would use the name of the property but as all lowercase and underscore separated, as well as the C# style property name. 
 
The project Model.cs needed a lot of new things that just isn't covered by scgen, and I didn't want to add dedicated properties to scgen that only this project needs.
 
So, my solution...
 
Long story short, Go allows you to deserialize json to a concrete structure much like Newtonsoft.Json, and it allows a field type to be json.RawMessage. So the FieldType type looks like this now
type FieldType struct {
	TypeName      string          `json:"typeName"`
	CodeType      string          `json:"codeType"`
	Suffix        string          `json:"suffix"`
	PropertiesRaw json.RawMessage `json:"properties"`
	Properties    FieldTypePropertyMap
}

type FieldTypePropertyMap map[string]string

To define these properties, you can just add a "properties" json property to the field type. Here's an example for the "checkbox" field type:

        { "typeName": "Checkbox", "codeType": "bool", "suffix": "", "properties": {"SpecialProperty": "SpecialPropertyValue"} },

The process for including this and processing the text template is pretty verbose. It's not so bad if you can always depend on that property being there. But first, if you need to check for the property, you can use Go's Text Template "index" method, like so:

{{ if ne (index $field.Properties "propertyName") "" }}   The property exists, use it here.  {{ end }}

If the map doesn't include that property, it will just return a blank string. And to write it out in the template, simply do this, again with the index method:

{{ index $field.Properties "propertyName" }}

Feel free to browse the code at github.com/jasontconnell/scgen.  There was also an update to the configuration helper that I use for nearly every project I create in Go, located at github.com/jasontconnell/conf.

Hybrid Car Shopping

Yes, I'm in the market for a new car. My car might last a few more years, it only has 70,000 miles on it.

I've been saying for the past few years, the next new car I buy would be a Tesla. However, there are many factors leading me away from that. First, they're freakin' expensive. Second, I would have to install something at my house to get the Level II charging which takes the charge time from something like days to overnight. Third, I'm not sure right now is the right time.

So instead of Tesla, I've been looking at 2019 Hybrid models, like the Kia Niro, Hyundai Ioniq, and the Honda Insight. They all seem like good cars. However, I want a hatchback, since I currently drive a hatchback, so the Insight loses a point, and I want 55 mpg. So the final choice would be the Ioniq. The choice between the Insight and the Ioniq will be left up to test drives of each, but I hope the Ioniq is good enough, because I want that car :)

So, I was doing some math on the choice to go hybrid. The one constant drive in my life is to and from work. The trip is 14 miles round trip. I've measured the MPG for my car, the 2011 Hyundai Elantra Touring. It has a 14 gallon tank. It has been getting 20 mpg for the trip, which is highly reminiscent of city driving. The Ioniq and Insight both advertise 55mpg city.

14 miles per day, figure in 4 weeks of vacation (not figuring in working from home some days). Figure 240 working days a year where I'm making the 14 mile round trip, so 3360 miles. Just for work.

At my current car rate, I will use 168 gallons of fuel per year. Again, this is just counting as if all I ever used my car for was back and forth to work, nothing else. With a gas tank of 14 gallons, this would require a fillup 12 times a year.

At 55mpg city, I will use 61 gallons of fuel, requiring a fill up of (on the Ioniq's 11.9 gallon tank) 5 times per year.

Gas prices change so I will just measure in fillups per year. I've owned my car since May or June of 2012, so 6 and a half years. I bought it with 8K miles on it. So I've put on 62,000 miles. I don't know how many times I've filled up, but I would wager that I've only averaged maybe 22-25 miles per gallon. We'll be generous and call it 25.

By that number I've used 2,480 gallons of gas, or have filled up 177 times. By the same token, if I got the Ioniq, and it kept close to advertised mpg (or, let's be less generous and say it *only* ends up getting 45 mpg), after 62,000 miles, I would have used 1,377 gallons of gas, and would have filled up 115 times. Here the fill up number is misleading, at that scale a 2.1 gallon difference, or 15% smaller tank, makes a big difference, I would use over 1000 gallons less.

At $2.50 current gas price, I would save $2,500 over the 6.5 years I've owned the car.

So that's not a lot of money. However, I'm still going to get it!  I need a new car. I joked with some people, the 2011 Elantra Touring is falling apart around me, in a few years I'll just be driving a frame around :)  The car works, it gets me places, however there's nothing cool about it. Ok, it's got roof racks. I'll miss those. If I really wanted to keep them I would seriously consider the Kia Niro. But the Ioniq has a .24 coefficient of drag. 55 city, 54 highway! That's pretty awesome. It pretty much matches up in tech against the Insight, however, the Insight has that sweet electronic parking brake, while the Ioniq has just the foot brake. My Elantra even has a hand brake!  Oh well.