2009-10-22
Optimizing Tortoise SVN Cache (TSVNCache.exe) disk io
Since I got my new computer with an SSD inside, disk io of the TSVNCache.exe process hasn't really been much of a problem. However, I sometimes need to do some stuff on my old computer, as not everything have been moved as of yet, and TSVNCache.exe really makes it sluggish. Today I decided to try to do something about it, and I found this page on how to optimize disk io of Tortoise SVN Cache .
2009-10-21
Pausing in a batch script
While debugging my hybrid dos/python invokation via psexec, I needed a way to pause the windows batch script for while (I tried to figure out the path of the file once psexec had copied it to the remote computer). I found this way of doing it, which seems to work on both Windows Server 2003 and Windows Server 2008.
@ping -n 30 -w 1000 0.0.0.1 > NUL
Executing python script via psexec
Had to do some file copying on a couple of servers today, involving quite a few directories and files. It seemed like a really boring/repetitive task to do manually, begging to be automated. As usual when there are many steps to be executed and a human involved (or, atleast when I'm involved) it is quite easy to miss something and create a mess out of it.
The problem is that those servers are running Windows, so there is only one way that I know of to automate the file copying, by using psexec from the pstools suite to execute a script on the list of servers. A windows batch-script would probably have worked, however, as there was quite a few steps involved, and I'm not really confident in writing batch-scripts, I decided to try to use a python script for the file copying instead.
The problem with psexec is that it can only copy one file at a time to be executed remotely, and as python was not in the %PATH of the servers I had to find a way to make psexec copy the script, and run it, and then run the python script. One way would of course be to have the python script as a separate file, but that would mean I had to copy it separately from the psexec invocation.
Instead I decided to try to embed the python program inside the batch script, thereby creating a hybrid batch/python program, a file which is valid both for the batch script and for the python interpreter. A quick search revealed this post, which contained an embryo for the hybrid. Adapting it worked quite nicely while running locally.
I saved this as test.bat and tried to invoke it with the following command:
Psexec runs the script fine, but python fails to find test.bat. After some debugging, it turned out that the test.bat file the psexec copies to the server ends up in c:\windows\system32 directory. However, I really could not get python to execute the script, and I have no idea why.
After some time banging my head against the wall, and littering the script with various attempts at invoking python in strange ways, was to pipe the script to the python-interpreter. This works like a charm. Full solution:
The problem is that those servers are running Windows, so there is only one way that I know of to automate the file copying, by using psexec from the pstools suite to execute a script on the list of servers. A windows batch-script would probably have worked, however, as there was quite a few steps involved, and I'm not really confident in writing batch-scripts, I decided to try to use a python script for the file copying instead.
The problem with psexec is that it can only copy one file at a time to be executed remotely, and as python was not in the %PATH of the servers I had to find a way to make psexec copy the script, and run it, and then run the python script. One way would of course be to have the python script as a separate file, but that would mean I had to copy it separately from the psexec invocation.
Instead I decided to try to embed the python program inside the batch script, thereby creating a hybrid batch/python program, a file which is valid both for the batch script and for the python interpreter. A quick search revealed this post, which contained an embryo for the hybrid. Adapting it worked quite nicely while running locally.
rem = """-*-Python-*- script
@echo off
rem -------------------- DOS section --------------------
c:\python25\python %0 %*
goto exit
"""
# -------------------- Python section --------------------
print "hi from python"
DosExitLabel = """
:exit
rem """
I saved this as test.bat and tried to invoke it with the following command:
psexec \\computer -u <username> -s -c -f test.bat
Psexec runs the script fine, but python fails to find test.bat. After some debugging, it turned out that the test.bat file the psexec copies to the server ends up in c:\windows\system32 directory. However, I really could not get python to execute the script, and I have no idea why.
After some time banging my head against the wall, and littering the script with various attempts at invoking python in strange ways, was to pipe the script to the python-interpreter. This works like a charm. Full solution:
rem = """-*-Python-*- script
@echo off
rem -------------------- DOS section --------------------
type %~f0 | c:\python25\python - %*
goto exit
"""
# -------------------- Python section --------------------
print "hi from python"
DosExitLabel = """
:exit
rem """
2009-10-10
Installing Windows 7 on Lenovo Thinkpad T500
Got a new laptop two weeks ago, a Lenovo Thinkpad T500 with P8400 processor running at 2.4 GHz, 15" non-led tft at 1600x1050 and switchable graphics (Intel 4500 integrated and Radeon HD3650). It seems really cool running, which was my number one complaint with my old laptop (3 year old HP nc8430/T7200@2GHz/Radeon X1600). Although, I should point out that I haven't used the radeon graphics of the Thinkpad yet. It could suffer from the same problems that plagues the HP, if you use an external monitor, the Radeon automatically kicks into "high performance mode", making it extremely hot and therefor very noisy. See this thread on HP's business support forums for more info.
Other than installing Windows 7 on it I haven't had much time to mess around with it (it's been alot of work the last couple of weeks). Anyway, two days ago I finally got the last missing pieces for the new setup (2 GB extra DDR3-memory) and decided to finally get around to installing the Intel X25-M G2 inside it.
Although I didn't have much installed on the harddrive, I had already figured out and installed the correct Win7 drivers on it. To spare me the boring job of doing this all over again, I opted to use Clonezilla to copy the contents of the harddrive to the new SSD.
Only problem was that the built-in drive was 320GB and the SSD only 160GB. Due to this, Clonezilla wouldn't let me clone the partition as it stood. Scratching my head for a while, I googled and found the ntfsresize utility. After issuing the command "ntfsresize -s 100G /dev/sda1" I started waiting. And waiting. I switched console and ran top, noticing ntfsresize drawing 100% cpu with nearly no IO. A good 30 minutes after starting it, I interrupted it and decided to re-install Windows all over again.
I powered off the computer, disassembled the "cage" for the harddrive and installed the ssd instead. Booted up and installed Windows 7, and the drivers (which was quite quick, now that I had sorted out which ones were necessary). Then I booted with clonezilla once again, to take a backup of the freshly installed system, in case I should screw something up during install of the necessary programs.
These are the Lenovo Thinkpad T500 Windows 7 drivers I wound up installing:
Other than installing Windows 7 on it I haven't had much time to mess around with it (it's been alot of work the last couple of weeks). Anyway, two days ago I finally got the last missing pieces for the new setup (2 GB extra DDR3-memory) and decided to finally get around to installing the Intel X25-M G2 inside it.
Although I didn't have much installed on the harddrive, I had already figured out and installed the correct Win7 drivers on it. To spare me the boring job of doing this all over again, I opted to use Clonezilla to copy the contents of the harddrive to the new SSD.
Only problem was that the built-in drive was 320GB and the SSD only 160GB. Due to this, Clonezilla wouldn't let me clone the partition as it stood. Scratching my head for a while, I googled and found the ntfsresize utility. After issuing the command "ntfsresize -s 100G /dev/sda1" I started waiting. And waiting. I switched console and ran top, noticing ntfsresize drawing 100% cpu with nearly no IO. A good 30 minutes after starting it, I interrupted it and decided to re-install Windows all over again.
I powered off the computer, disassembled the "cage" for the harddrive and installed the ssd instead. Booted up and installed Windows 7, and the drivers (which was quite quick, now that I had sorted out which ones were necessary). Then I booted with clonezilla once again, to take a backup of the freshly installed system, in case I should screw something up during install of the necessary programs.
These are the Lenovo Thinkpad T500 Windows 7 drivers I wound up installing:
- ricoh multicard reader - got rid of 2 unknown devices
- system interface - needed for hotkeys I believe
- hotkey
- power management driver and power manager
- management engine and serial over lan - got rid of 1 or 2 unknown devices
- ultranav1 and ultranav utility
- switchable graphics
- easyeject driver/application - I haven't yet installed this one, not sure that I need it
2009-02-21
Setting up rdiff-backup 1.2.5 on Ubuntu Hardy 8.04
I wanted to backup a server running Debian Lenny 5.0, to a slightly older box running Ubuntu Hardy 8.04 using rdiff-backup. As rdiff-backup complains if the version-numbers on both sides are not the same I had to upgrade the older box to version 1.2.5 of rdiff-backup.
I have previously used standard rsync for efficiently mirror the contents of a server to the backup server. This meant that I also had to change my backupscript slightly, to accomodate rdiff-backup usage.
aptitude install librsync-dev python-dev
tar xzvf tar xzvf rdiff-backup-1.2.6.tar.gz
cd rdiff-backup-1.2.6
python setup.py install --prefix=/usr/local
I have previously used standard rsync for efficiently mirror the contents of a server to the backup server. This meant that I also had to change my backupscript slightly, to accomodate rdiff-backup usage.
#!/bin/sh
EXCLUDE="/var/lib/mysql/ /tmp/ /proc/ /sys/ /dev/ /bin/ /sbin/ /lib/ /usr/"
INCLUDE="/usr/local/"
exclude=""
for i in $EXCLUDE; do
exclude="$exclude --exclude $i"
done
include=""
for i in $INCLUDE; do
include="$include --include $i"
done
# rsync --archive --delete --sparse $exclude $include --rsh=ssh / backupuser@backupserver:root
rdiff-backup $include $exclude / backupuser@backupserver::/home/backupuser/rdiff-backup/
2009-01-21
How to recycle an Application Pool on Windows Server 2008 from command line
For a client I wanted to implement automatic recycling of an Application Pool on Windows Server 2008, to be called from an event handler in nagios. I scorched the net for possible solutions, and found many suggestions, including iisreset.exe, iisapp.vbs and various wmi-based scripting techniques, none of which worked out.
After a while, nearly giving up, I stumbled upon this page which showed the correct technique:
After a while, nearly giving up, I stumbled upon this page which showed the correct technique:
c:\windows\system32\inetsrv\appcmd.exe recycle apppool "website.com"
2008-12-10
Difference between Mocks and Stubs
Someone new to unit-testing, as myself, might have a hard time judging when to use the various techniques available at your disposal. One of the choices to make is when to use a mock-object and when to use a stub.
After having spent an evening trying to iron out the issues and coming up with a clearer understanding, these are my thoughts.
The legendary Martin Fowler have written a good article about this. The article suggest the word double as a generic form of objects which are utilized in testing scenarios in place of the real implementation. Stubs and mocks are therefore specific versions of doubles. A stub is described as primarily "returning canned answers" and secondarily remembering what was called, while a mock are pre-set with expectations about which calls should be made.
This means that stubs don't really care whether they were called or not. The mock object, on the other hand, enforces which calls should be made, in what order and with what parameters. Mocks are concerned with the behaviour, in addition to the end-result, whereas stubs are only concerned with the end-result. Stubs lends itself to state-based testing, given A,B,C perform X, expect A',B',C'. Mocks on the other hand also enforces how the end-state was derived at.
One could argue that as mocks dictates the behaviour of a method, they break encapsulation. However this is a double edged sword, as it can be used to flesh out behaviour in an iterative manner, exemplified by Behaviour-Driven-Development (BDD).
In the context of Rhino Mocks, the waters are muddied. From this page we gather that you can set expectations on stubs, but they will never be enforced, and will therefore never make a test fail. On the other hand, you can use the
Further, in Rhino Mocks, when you create a stub of an object, all properties functions as you would normally expect them to (if you set a value X, you will get back X later on).
After having spent an evening trying to iron out the issues and coming up with a clearer understanding, these are my thoughts.
The legendary Martin Fowler have written a good article about this. The article suggest the word double as a generic form of objects which are utilized in testing scenarios in place of the real implementation. Stubs and mocks are therefore specific versions of doubles. A stub is described as primarily "returning canned answers" and secondarily remembering what was called, while a mock are pre-set with expectations about which calls should be made.
This means that stubs don't really care whether they were called or not. The mock object, on the other hand, enforces which calls should be made, in what order and with what parameters. Mocks are concerned with the behaviour, in addition to the end-result, whereas stubs are only concerned with the end-result. Stubs lends itself to state-based testing, given A,B,C perform X, expect A',B',C'. Mocks on the other hand also enforces how the end-state was derived at.
One could argue that as mocks dictates the behaviour of a method, they break encapsulation. However this is a double edged sword, as it can be used to flesh out behaviour in an iterative manner, exemplified by Behaviour-Driven-Development (BDD).
In the context of Rhino Mocks, the waters are muddied. From this page we gather that you can set expectations on stubs, but they will never be enforced, and will therefore never make a test fail. On the other hand, you can use the
Repeat.Any()method on a mocked call to basically make it behave like a stubbed method (if it was called it is ok, but if not that's ok too).
Further, in Rhino Mocks, when you create a stub of an object, all properties functions as you would normally expect them to (if you set a value X, you will get back X later on).
Subscribe to:
Posts (Atom)