tag:blogger.com,1999:blog-10348362364049252182024-03-13T03:01:47.661-03:00EE & CSElectronics Engineering and Computer ScienceMarcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-1034836236404925218.post-66030399814416276322017-10-24T15:50:00.002-02:002017-11-19T12:23:12.773-02:00Stop IPTables and Other Kernel Messages From Flooding Your Console<h2>
The Problem</h2>
Console gets flooded with messages like this:<br />
<span style="font-family: monospace;"><span style="background-color: white; color: #18b218;"><br /></span></span>
<span style="font-family: monospace;"><span style="background-color: white; color: #18b218;">[1909322.686129] </span><span style="background-color: white; font-weight: bold;">SFW2-INext-ACC-TCP IN=eth0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx SRC=xxx.xxx.xxx.xxx DST=xxx.xxx.xxx.xxx LEN=60 TOS=0x00 PREC=0x00</span><br /> TTL=39 ID=30338 PROTO=TCP SPT=33453 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)</span><br />
<span style="font-family: monospace;"><br /></span>
<br />
<h2>
<span style="font-family: inherit;">Solutions</span></h2>
<span style="font-family: inherit;">They all rely on something similar to writing to /proc/sys/kernel/printk:</span><br />
<br />
<ol>
<li><span style="font-family: "courier new" , "courier" , monospace;">$ <span style="background-color: white;">echo "3 4 1 3" > /proc/sys/kernel/printk</span></span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">$ sysctl -w kernel.printk="3 4 1 3"</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">$ dmesg -n 4</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Create the file /etc/sysctl.d/50-stop-dmesg-flood.conf and put the following line in it: kernel.printk = 3 4 1 3</span></li>
</ol>
<br />
<div>
<br /></div>
<div>
To check:</div>
<div>
<ol>
<li><span style="font-family: "courier new" , "courier" , monospace;">$ cat /proc/sys/kernel/printk</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">$ sysctl kernel.printk</span></li>
</ol>
</div>
<h2>
References</h2>
<div>
<ol>
<li><a href="https://superuser.com/questions/351387/how-to-stop-kernel-messages-from-flooding-my-console">https://superuser.com/questions/351387/how-to-stop-kernel-messages-from-flooding-my-console</a></li>
</ol>
</div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com1tag:blogger.com,1999:blog-1034836236404925218.post-68415537481709963892017-10-20T20:18:00.000-02:002018-01-12T11:54:10.425-02:00Google chrome version 62.0.3202.62 is not working for me<h2 class="tr_bq">
The Problem</h2>
<div>
Google chrome version 62.0.3202.62 starts but is not working in several linux distributions. It shows lots of sad faces in each open tab, all of the chrome extensions crash. Not even "Help -> About Google Chrome" works.</div>
<div>
<br /></div>
<h2>
The "Not Really Good Solution"</h2>
<div>
You are able to start it if you run:</div>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">$ google-chrome --no-sandbox &</span></blockquote>
At least you can quickly see some important information you left in a tab before you did the upgrade.<br />
<br />
<h2>
The Slightly Better Solution</h2>
<div>
This one works too:</div>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">$ <span style="background-color: white;">google-chrome --disable-namespace-sandbox &</span></span></blockquote>
Better than no sandbox, I guess...<br />
<br />
<h2>
Good links for information</h2>
I'll try to keep them here:<br />
<ol>
<li><a href="https://bugs.chromium.org/p/chromium/issues/detail?id=776359">https://bugs.chromium.org/p/chromium/issues/detail?id=776359</a></li>
<li><a href="https://forums.opensuse.org/showthread.php/527688-Google-Chrome-not-working-after-update">https://forums.opensuse.org/showthread.php/527688-Google-Chrome-not-working-after-update</a></li>
<li><a href="https://www.reddit.com/r/SolusProject/comments/778gcf/google_chrome_stable_busted_after_update_to_62/" target="_blank">https://www.reddit.com/r/SolusProject/comments/778gcf/google_chrome_stable_busted_after_update_to_62/</a></li>
</ol>
<br />
<h2>
Google Chrome Legacy Versions, no RPMS unfortunately</h2>
<div>
<ol>
<li><a href="https://www.slimjet.com/chrome/google-chrome-old-version.php" target="_blank">https://www.slimjet.com/chrome/google-chrome-old-version.php</a></li>
<li><a href="https://www.ubuntuupdates.org/pm/google-chrome-stable">https://www.ubuntuupdates.org/pm/google-chrome-stable</a></li>
<li><a href="https://archive.org/download/Google-Chrome-Legacy-Versions" target="_blank">https://archive.org/download/Google-Chrome-Legacy-Versions</a></li>
</ol>
<span id="goog_1933757830"></span></div>
<br />
<h2>
</h2>
<div>
<span style="font-family: monospace;"><span style="background-color: white;">
</span></span>
</div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com1tag:blogger.com,1999:blog-1034836236404925218.post-81687694092000842762016-10-24T15:31:00.000-02:002017-05-03T16:07:51.710-03:00Why is ssh slow to connect?<h2>
<span style="font-family: inherit;">Introduction</span></h2>
<span style="font-family: inherit;">There are several reasons why ssh may be getting slow to connect. I'll cover here the ones that happened to me.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<h2>
<span style="font-family: inherit;">On the server side</span></h2>
<ol>
<li><span style="font-family: inherit;">Make sure you have a line "UseDNS no" uncommented.</span></li>
<li><span style="font-family: inherit;">Make sure you have "good" (i.e., reachable) DNS servers in /etc/resolv.conf.</span></li>
<li><span style="font-family: inherit;">Make sure you have no reverse DNS lines in /etc/hosts.deny.</span></li>
</ol>
<div>
<span style="font-family: inherit;">The server side configuration is typically in the file /etc/ssh/<span style="background-color: white;">sshd_config. </span></span><span style="background-color: white; font-family: inherit;">Every time you make a change to this file, remember to restart the ssh daemon ("# systemctl restart sshd").</span><br />
<br />
<ol></ol>
</div>
<div>
Item number 3 deserves some comments, since it was the hardest to get right. <a href="http://denyhosts.sourceforge.net/" target="_blank">Denyhosts</a> may be adding lines like <span style="background-color: white;"> </span><span style="background-color: white;">"xxx-xxx-xxx-xxx.rev.abxtelex.eu" that will trigger reverse DNS even if you have "UseDNS no" in the configuration file. You have to remove these lines, but if you have Denyhosts installed, maybe you <span style="font-family: inherit;">already know how hard it is to remove entries from this file, since they keep reappearing. </span></span><span style="font-family: inherit;"><span style="background-color: white;">If you use OpenSUSE, you have a script called "/usr/sbin/</span><span style="background-color: white;">dh_reenable" that will do the trick for you. If you are not using OpenSUSE or have installed Denyhosts by hand, then you have to do it manually, take a look at the <a href="http://denyhosts.sourceforge.net/faq.html#3_19" target="_blank">Denyhosts FAQ here</a>.</span></span></div>
<div>
<span style="background-color: white;"><br /></span></div>
<div>
<span style="background-color: white;">I still don't have a good solution to problem number 3, the best thing would be that Denyhosts did not add reverse DNS entries in /etc/resolv.conf, but I did not find a way to configure it to behave like that.</span></div>
<div>
<span style="background-color: white;"><br /></span></div>
<br />
<h2>
<span style="font-family: inherit;">On the client side</span></h2>
<span style="font-family: inherit;">The global client side configuration is typically in the file /etc/ssh/<span style="background-color: white;">ssh_config. But you can configure things on a user level by editing the file ~/.ssh/config.</span><span style="background-color: white;"><br /></span></span><br />
<span style="background-color: white;"><span style="font-family: inherit;">I use the local configuration file like this:</span></span><br />
<span style="background-color: white;"><span style="font-family: inherit;"><br /></span></span>
<span style="font-family: inherit;">Host *</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>Compression yes</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>ForwardX11=yes</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>ForwardX11Trusted=yes</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>GSSAPIAuthentication=no</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Host analise</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>HostName 192.168.1.1</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Host home</span><br />
<span style="font-family: inherit;"><span class="Apple-tab-span" style="white-space: pre;"> </span>Hostname xxxx.noip.me</span><br />
<span style="font-family: inherit;"><span style="background-color: white;"></span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">That way I can just type "ssh analise" or "ssh home". In the "home" case, it has the advantage of making my dynamic dns name shorter.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<ol>
<li><span style="font-family: inherit;">Make sure you have the line "GSSAPIAuthentication=no".</span></li>
</ol>
<div>
<br /></div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-70923176123671156382016-08-01T16:06:00.000-03:002016-08-01T16:06:48.942-03:00OpenSUSE Leap 42.1 and nvidia kernel driver<h2>
The symptoms</h2>
Nvidia kernel drivers were not loaded after kernel update.<br />
<br />
<h2>
The problem</h2>
The package nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64 runs a script that regenerates the kernel drivers and creates symlinks in /lib/modules/4.1.27-27-default/weak-updates/updates. After running:<br />
<br />
<b><i># zypper in --force $(rpm -qa "nvidia-gfx*kmp*")</i></b><br />
<b><i><br /></i></b>
I noticed that the links were not beeing generated:<br />
<br />
<b><i># zypper in --force $(rpm -qa "nvidia-gfx*kmp*")</i></b><br />
<b><i>Retrieving repository 'network:utilities' metadata ..........................................................................................................[done]</i></b><br />
<b><i>Building repository 'network:utilities' cache ...............................................................................................................[done]</i></b><br />
<b><i>Loading repository data...</i></b><br />
<b><i>Reading installed packages...</i></b><br />
<b><i>Forcing installation of 'nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64' from repository 'nVidia Graphics Drivers'.</i></b><br />
<b><i>Resolving package dependencies...</i></b><br />
<b><i><br /></i></b>
<b><i>The following package is going to be reinstalled:</i></b><br />
<b><i> nvidia-gfxG04-kmp-default</i></b><br />
<b><i><br /></i></b>
<b><i>1 package to reinstall.</i></b><br />
<b><i>Overall download size: 5.8 MiB. Already cached: 0 B. No additional space will be used or freed after the operation.</i></b><br />
<b><i>Continue? [y/n/? shows all options] (y): </i></b><br />
<b><i>Retrieving package nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64 (1/1), 5.8 MiB ( 64.6 MiB unpacked)</i></b><br />
<b><i>Retrieving: nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64.rpm ..........................................................................[done (2.2 MiB/s)]</i></b><br />
<b><i>Checking for file conflicts: ................................................................................................................................[done]</i></b><br />
<b><i>(1/1) Installing: nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64 ....................................................................................[done]</i></b><br />
<b><i>Additional rpm output:</i></b><br />
<b><i>make: Entering directory '/usr/src/linux-4.1.27-27-obj/x86_64/default'</i></b><br />
<b><i> Building modules, stage 2.</i></b><br />
<b><i> MODPOST 0 modules</i></b><br />
<b><i>make: Leaving directory '/usr/src/linux-4.1.27-27-obj/x86_64/default'</i></b><br />
<b><i>/usr/src/kernel-modules/nvidia-367.35-default /</i></b><br />
<b><i>make "CC=cc" KBUILD_OUTPUT=/usr/src/linux-obj/x86_64/default KBUILD_VERBOSE= -C /lib/modules/4.1.27-27-default/source M=/usr/src/kernel-modules/nvidia-367.35-default ARCH=x86_64 NV_KERNEL_SOURCES=/lib/modules/4.1.27-27-default/source NV_KERNEL_OUTPUT=/usr/src/linux-obj/x86_64/default NV_KERNEL_MODULES="nvidia nvidia-uvm nvidia-modeset nvidia-drm" INSTALL_MOD_DIR=kernel/drivers/video modules</i></b><br />
<b><i>make[1]: Entering directory '/usr/src/linux-4.1.27-27'</i></b><br />
<b><i>make[2]: Entering directory '/usr/src/linux-4.1.27-27-obj/x86_64/default'</i></b><br />
<b><i> Building modules, stage 2.</i></b><br />
<b><i> MODPOST 4 modules</i></b><br />
<b><i>make[2]: Leaving directory '/usr/src/linux-4.1.27-27-obj/x86_64/default'</i></b><br />
<b><i>make[1]: Leaving directory '/usr/src/linux-4.1.27-27'</i></b><br />
<b><i>ld -T /lib/modules/4.1.27-27-default/source/scripts/module-common.lds -r -o nv-linux.o \</i></b><br />
<b><i> nvidia.mod.o nvidia/nv-interface.o</i></b><br />
<b><i>/</i></b><br />
<b><i><br /></i></b>
<b><i>Modprobe blacklist files have been created at /etc/modprobe.d to prevent Nouveau from loading. This can be reverted by deleting /etc/modprobe.d/nvidia-*.conf.</i></b><br />
<b><i><br /></i></b>
<b><i>*** Reboot your computer and verify that the NVIDIA graphics driver can be loaded. ***</i></b><br />
<b><i><br /></i></b>
<b><i>depmod: WARNING: //lib/modules/4.1.27-27-default/misc/vboxvideo.ko disagrees about version of symbol VBoxGuest_RTLogBackdoorPrintf</i></b><br />
<b><i>depmod: WARNING: //lib/modules/4.1.27-27-default/misc/vboxvideo.ko disagrees about version of symbol VBoxGuest_RTErrConvertToErrno</i></b><br />
<b><i>depmod: WARNING: //lib/modules/4.1.27-27-default/misc/vboxvideo.ko disagrees about version of symbol VBoxGuest_RTAssertShouldPanic</i></b><br />
<b><i>depmod: WARNING: //lib/modules/4.1.27-27-default/misc/vboxvideo.ko disagrees about version of symbol VBoxGuest_RTAssertMsg1Weak</i></b><br />
<b><i>depmod: WARNING: //lib/modules/4.1.27-27-default/misc/vboxvideo.ko disagrees about version of symbol VBoxGuest_RTAssertMsg2Weak</i></b><br />
<b><i>Warning: /lib/modules/4.1.27-27-default is inconsistent</i></b><br />
<b><i>Warning: weak-updates symlinks might not be created</i></b><br />
<b><i><br /></i></b>
<b><i><br /></i></b>
<b><i>Output of nvidia-gfxG04-kmp-default-367.35_k4.1.12_1-25.1.x86_64.rpm %posttrans script:</i></b><br />
<b><i> Creating initrd: /boot/initrd-4.1.12-1-default</i></b><br />
<b><i> Executing: /usr/bin/dracut --logfile /var/log/YaST2/mkinitrd.log --force --force-drivers "nvidia -drm" /boot/initrd-4.1.12-1-default 4.1.12-1-default</i></b><br />
<b><i> *** Including module: bash ***</i></b><br />
<b><i> *** Including module: warpclock ***</i></b><br />
<b><i> *** Including module: i18n ***</i></b><br />
<b><i> *** Including module: ifcfg ***</i></b><br />
<b><i> *** Including module: drm ***</i></b><br />
<b><i> *** Including module: plymouth ***</i></b><br />
<b><i> *** Including module: kernel-modules ***</i></b><br />
<b><i> Omitting driver i2o_scsi</i></b><br />
<b><i> *** Including module: resume ***</i></b><br />
<b><i> *** Including module: rootfs-block ***</i></b><br />
<b><i> *** Including module: terminfo ***</i></b><br />
<b><i> *** Including module: udev-rules ***</i></b><br />
<b><i> Skipping udev rule: 91-permissions.rules</i></b><br />
<b><i> Skipping udev rule: 80-drivers-modprobe.rules</i></b><br />
<b><i> *** Including module: haveged ***</i></b><br />
<b><i> *** Including module: systemd ***</i></b><br />
<b><i> *** Including module: usrmount ***</i></b><br />
<b><i> *** Including module: base ***</i></b><br />
<b><i> *** Including module: fs-lib ***</i></b><br />
<b><i> *** Including module: shutdown ***</i></b><br />
<b><i> *** Including module: suse ***</i></b><br />
<b><i> *** Including modules done ***</i></b><br />
<b><i> *** Installing kernel module dependencies and firmware ***</i></b><br />
<b><i> *** Installing kernel module dependencies and firmware done ***</i></b><br />
<b><i> *** Resolving executable dependencies ***</i></b><br />
<b><i> *** Resolving executable dependencies done***</i></b><br />
<b><i> *** Hardlinking files ***</i></b><br />
<b><i> *** Hardlinking files done ***</i></b><br />
<b><i> *** Stripping files ***</i></b><br />
<b><i> *** Stripping files done ***</i></b><br />
<b><i> *** Generating early-microcode cpio image ***</i></b><br />
<b><i> *** Constructing GenuineIntel.bin ****</i></b><br />
<b><i> *** Store current command line parameters ***</i></b><br />
<b><i> Stored kernel commandline:</i></b><br />
<b><i> rd.driver.pre=nvidia</i></b><br />
<b><i> rd.driver.pre=_drm</i></b><br />
<b><i> resume=UUID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</i></b><br />
<b><i> root=UUID=xxxxxxxxxxxxxxxxxxxxxx rootflags=rw,relatime,data=ordered rootfstype=ext4</i></b><br />
<b><i> *** Creating image file ***</i></b><br />
<b><i> *** Creating image file done ***</i></b><br />
<b><i> Some kernel modules could not be included</i></b><br />
<b><i> This is not necessarily an error:</i></b><br />
<b><i> swap</i></b><br />
<b><i> _drm</i></b><br />
<b><i> Update bootloader...</i></b><br />
<b><i> Creating initrd: /boot/initrd-4.1.27-24-default</i></b><br />
<b><i> Executing: /usr/bin/dracut --logfile /var/log/YaST2/mkinitrd.log --force --force-drivers "nvidia -drm" /boot/initrd-4.1.27-24-default 4.1.27-24-default</i></b><br />
<b><i> *** Including module: bash ***</i></b><br />
<b><i> *** Including module: warpclock ***</i></b><br />
<b><i> *** Including module: i18n ***</i></b><br />
<b><i> *** Including module: ifcfg ***</i></b><br />
<b><i> *** Including module: drm ***</i></b><br />
<b><i> *** Including module: plymouth ***</i></b><br />
<b><i> *** Including module: kernel-modules ***</i></b><br />
<b><i> Omitting driver i2o_scsi</i></b><br />
<b><i> *** Including module: resume ***</i></b><br />
<b><i> *** Including module: rootfs-block ***</i></b><br />
<b><i> *** Including module: terminfo ***</i></b><br />
<b><i> *** Including module: udev-rules ***</i></b><br />
<b><i> Skipping udev rule: 91-permissions.rules</i></b><br />
<b><i> Skipping udev rule: 80-drivers-modprobe.rules</i></b><br />
<b><i> *** Including module: haveged ***</i></b><br />
<b><i> *** Including module: systemd ***</i></b><br />
<b><i> *** Including module: usrmount ***</i></b><br />
<b><i> *** Including module: base ***</i></b><br />
<b><i> *** Including module: fs-lib ***</i></b><br />
<b><i> *** Including module: shutdown ***</i></b><br />
<b><i> *** Including module: suse ***</i></b><br />
<b><i> *** Including modules done ***</i></b><br />
<b><i> *** Installing kernel module dependencies and firmware ***</i></b><br />
<b><i> *** Installing kernel module dependencies and firmware done ***</i></b><br />
<b><i> *** Resolving executable dependencies ***</i></b><br />
<b><i> *** Resolving executable dependencies done***</i></b><br />
<b><i> *** Hardlinking files ***</i></b><br />
<b><i> *** Hardlinking files done ***</i></b><br />
<b><i> *** Stripping files ***</i></b><br />
<b><i> *** Stripping files done ***</i></b><br />
<b><i> *** Generating early-microcode cpio image ***</i></b><br />
<b><i> *** Constructing GenuineIntel.bin ****</i></b><br />
<b><i> *** Store current command line parameters ***</i></b><br />
<b><i> Stored kernel commandline:</i></b><br />
<b><i> rd.driver.pre=nvidia</i></b><br />
<b><i> rd.driver.pre=_drm</i></b><br />
<b><i> resume=UUID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx</i></b><br />
<b><i> root=UUID=xxxxxxxxxxxxxxxxxxxxxxx rootflags=rw,relatime,data=ordered rootfstype=ext4</i></b><br />
<b><i> *** Creating image file ***</i></b><br />
<b><i> *** Creating image file done ***</i></b><br />
<b><i> Some kernel modules could not be included</i></b><br />
<b><i> This is not necessarily an error:</i></b><br />
<b><i> swap</i></b><br />
<b><i> _drm</i></b><br />
<b><i> Update bootloader...</i></b><br />
<b><i><br /></i></b>
<h2>
The solution</h2>
Uninstall the virtualbox guest packages: <span style="font-weight: 600;">virtualbox-guest-kmp-default, </span><span style="font-weight: 600;">virtualbox-guest-tools </span>and<span style="font-weight: 600;"> </span><span style="font-weight: 600;">virtualbox-guest-x11</span> and rerun # zypper in --force $(rpm -qa "nvidia-gfx*kmp*") as root. After that, the links are created as follows:<br />
<br />
<span style="font-family: monospace;"><span style="background-color: white;">/lib/modules/4.1.27-27-default/weak-updates/updates # l
</span><br />total 8
<br />drwxr-xr-x 2 root root 4096 Aug 1 15:41 <span style="background-color: white; color: #5454ff; font-weight: bold;">.</span><span style="background-color: white;">/
</span><br />drwxr-xr-x 4 root root 4096 Jul 21 19:03 <span style="background-color: white; color: #5454ff; font-weight: bold;">..</span><span style="background-color: white;">/
</span><br />lrwxrwxrwx 1 root root 51 Aug 1 15:41 <span style="background-color: white; color: #18b2b2;">nvidia-drm.ko</span><span style="background-color: white;"> -> /lib/modules/4.1.12-1-default/updates/nvidia-drm.ko
</span><br />lrwxrwxrwx 1 root root 55 Aug 1 15:41 <span style="background-color: white; color: #18b2b2;">nvidia-modeset.ko</span><span style="background-color: white;"> -> /lib/modules/4.1.12-1-default/updates/nvidia-modeset.ko
</span><br />lrwxrwxrwx 1 root root 51 Aug 1 15:41 <span style="background-color: white; color: #18b2b2;">nvidia-uvm.ko</span><span style="background-color: white;"> -> /lib/modules/4.1.12-1-default/updates/nvidia-uvm.ko
</span><br />lrwxrwxrwx 1 root root 47 Aug 1 15:41 <span style="background-color: white; color: #18b2b2;">nvidia.ko</span><span style="background-color: white;"> -> /lib/modules/4.1.12-1-default/updates/nvidia.ko</span><br />
</span><br />
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-36933233743827763362016-03-08T19:00:00.001-03:002020-08-07T11:08:17.594-03:00Arduino Shield for custom board CPLD programming and testing using pogo pins<h2>
Arduino Shield for custom board CPLD programming and testing using pogo pins</h2>
<br />
This post is just to show how I have used the Arduino JTAG programming hardware/software that I have discussed before.<br />
<br />
The idea was to have a setup where I could both program and test a CPLD based board.<br />
<br />
Lets see the photos:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9le6L-qlSQ7mE3idGyrVnavcHryEvc-IMw4HGfhTHrrcrqepSj11HvfVU7RoUV0_ibtpdVxJ9VkW79k4Vdp3PdxAfp1DanYnI8tkvOwX3D6lhzcgsdVTOg9D_IUpIoRi7RRMADbEu1l6K/s1600/shield1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9le6L-qlSQ7mE3idGyrVnavcHryEvc-IMw4HGfhTHrrcrqepSj11HvfVU7RoUV0_ibtpdVxJ9VkW79k4Vdp3PdxAfp1DanYnI8tkvOwX3D6lhzcgsdVTOg9D_IUpIoRi7RRMADbEu1l6K/s320/shield1.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The photo above shows the support for the board to be programmed with the pogo pins at the center, the board itself and the top shield. I have used two identical shield boards and have spaced them to give the pogo pins the proper vertical direction.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Some of the pads on the board to be tested are SMD, others are through hole. Of course, after it was assembled, I realized I should have left the pogos that go peek at through holes slightly higher that those that peek at SMD pads. That would have made the process of fitting the board much better.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
In the same holes, I have mounted the board support, which is a kind of "negative" of the board. It consists of two milled pcbs, with two concentric circles to give support to the board. If you look carefully to the left of the photo, you can see a small dent that is used to give the board the proper orientation.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDAueIx5wl7DS55TnQU_tlXvJiaImzdYNmfxg_x-xlBRRwsh-VY4eoNdwb5lv8-SbrIfFSUXnE_JRMorAzF9Khbtp9svS4N8DkG0gERF0XZAxHbi2JZnSYmTpulPZ9eqFaylmcv-24TI5U/s1600/shield2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDAueIx5wl7DS55TnQU_tlXvJiaImzdYNmfxg_x-xlBRRwsh-VY4eoNdwb5lv8-SbrIfFSUXnE_JRMorAzF9Khbtp9svS4N8DkG0gERF0XZAxHbi2JZnSYmTpulPZ9eqFaylmcv-24TI5U/s320/shield2.jpg" width="320" /></a></div>
<br />
In this next photo, we can see the toggle clamp device used to hold the board in place in action.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjiZG_ZW_XrTMH75X9mAxfitrB3LT-uWaFXtdj3VMCeoXLbU1g7ap7sd0XihgHASWu85HrZ-hum9EkYk29nZMpshyCMuAtI50CUTIsKfQ8E9TIUpEdCMUGbeOnLhdjcPEginc7Cq3olTpe/s1600/shield3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjiZG_ZW_XrTMH75X9mAxfitrB3LT-uWaFXtdj3VMCeoXLbU1g7ap7sd0XihgHASWu85HrZ-hum9EkYk29nZMpshyCMuAtI50CUTIsKfQ8E9TIUpEdCMUGbeOnLhdjcPEginc7Cq3olTpe/s320/shield3.jpg" width="320" /></a></div>
<br />
In the last photo, we can see the full stack: the Arduino at the ground, the two shield boards in the middle and the support with a board in it.<br />
<br />
Some details for those interested:<br />
<br />
<ul>
<li>The pogo pins were bough <a href="http://dev.dangerousprototypes.com/store/designer/details/ian/12/dirty-pogo-pins" target="_blank">here in Dangerous Prototypes dev store</a>, at an excelent price.</li>
<li>The holding tool is an <a href="https://www.adafruit.com/products/2457" target="_blank">Adafruit's toggle clamp - large size</a>. </li>
<li>The shield and the board were made in <a href="http://dirtypcbs.com/index.php" target="_blank">DirtyPCBS</a>, also an amazing price.</li>
</ul>
<br />
Hope you like it, comments are welcome!<br />
<br />
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01CZTLHGE&asins=B01CZTLHGE&linkId=8c1857ca60588eb18e0832e8180342f3&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01EWNUUUA&asins=B01EWNUUUA&linkId=e19ef6aaf4dabf1393d03cdf74e7a4d5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00RRAM7PM&asins=B00RRAM7PM&linkId=5790a31bad78a8b03db7b9bec95a2d27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01L0ZL8N6&asins=B01L0ZL8N6&linkId=a4bff4046c0cd1b378c4192698032515&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPROFU&asins=B00NBPROFU&linkId=e5c6b5fbf584035d7a8f9e9e18cf4e27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPRQXK&asins=B00NBPRQXK&linkId=eb85e4e24d180083a35b69b3512ca7d2&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B074N7T1WS&asins=B074N7T1WS&linkId=7faa9bcaefca4f81e657fded958a5bf5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B0755VKZB5&asins=B0755VKZB5&linkId=3840a612e8bfeb8336b1d16786c37d34&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-3727836304318729472016-03-05T22:46:00.000-03:002018-01-25T13:47:31.426-02:00Serial Buffer Size versus Effective Bit Rate of Arduino USB<h2>
Introduction</h2>
<div>
<br /></div>
I have recently just met a few of the "gotchas" related to serial programming and memory on the Arduino, and got a few lessons. I was debugging someone else's non-working code. Non-working for, apparently, no good reason. To make a long story short, the problem was that the program used a large amount of SRAM (static RAM) memory, in the form of strings. The Arduino Uno has 32 Kib of flash, but only 2 KiB of SRAM. That is why strings in the Arduino should be kept in flash memory to save the precious SRAM. To do so, you have to use the "F()" macro, so that the compiler does that for you.<br />
<br />
Figuring out the problem was not easy, since using "Serial.print()" without "F()" to debug would just make things worse in an unpredictable way. But at a certain point, I got it, and since then I tried my best to spare SRAM. That is when I started facing the problem of the size of the serial buffer.<br />
<br />
Serial communication on the Arduino has one big problem: there is no hardware flow control. That means that if you want reliable communication, you must implement your own flow control mechanism. Anything you come up with software, implies a greater overhead than what you would get with a hardware mechanism. But of course, using a larger reception buffer would minimize the problem. The larger the buffer, the smaller the number of times the flow control mechanism must work.<br />
<br />
The Arduino software has a default size of 64 bytes for the serial buffer. I wonder if that was enough, so I wrote some code to test it.<br />
<br />
<h2>
Some theory</h2>
<br />
Lets try to come up with a model. Linear models look interesting, for a start. Lets try the following: the time that a transfer takes \((\Delta t)\) is proportional to the number of bytes we want to transfer. If you consider a serial transmission with UART, eight data bits, one start bit and one stop bit, the time to transfer one byte is proportional to either ten times the inverse of the bit rate or some byte processing overhead \((O_{byte})\), whichever is greater. But since these bytes are transfered in blocks, we can imagine that the total transfer time also has an overhead component proportional to the number of blocks \((O_{block})\). In equations:<br />
<br />
\begin{equation}
\Delta t = NumBytes \cdot \max \left( \frac{10}{BitRate}, O_{byte}\right)
+ NumBlocks \cdot O_{block} \end{equation}
<br />
<br />
\begin{equation} \Delta t = NumBytes \cdot \max \left( \frac{10}{BitRate}, O_{byte}\right) + \frac{NumBytes}{BlockSize} \cdot O_{block} \end{equation}<br />
<br />
\begin{equation} \label{eqDeltaTFinal} \Delta t = NumBytes \cdot \left[ \max \left( \frac{10}{BitRate}, O_{byte}\right) + \frac{O_{block}}{BlockSize} \right]
\end{equation}<br />
<br />
\begin{equation} \frac{\Delta t}{NumBytes} = \max \left( \frac{10}{BitRate}, O_{byte}\right) + \frac{O_{block}}{BlockSize} \end{equation}<br />
<br />
\begin{equation} \frac{10}{EffectiveBitRate} = \max \left( \frac{10}{BitRate}, O_{byte}\right) + \frac{O_{block}}{BlockSize} \end{equation}<br />
<br />
Equation \ref{eqDeltaTFinal} shows two things:<br />
<br />
<ol>
<li>We can mitigate the block overhead using a larger block size.</li>
<li>We should try to keep the byte overhead less than 10 times the inverse of the bit rate.</li>
</ol>
<br />
In this work, I will estimate the byte overhead and the block overhead from the measure of the effective bit rate for various block sizes.<br />
<br />
<h2>
Arduino Bit Rates</h2>
<div>
<br /></div>
For the data to have some meaning, we will have to use an exact calculation of the Arduino bit rates. The formula is (for AVR's U2X bit = 1):<br />
<br />
\begin{equation}\label{eqnBitRate}BitRate = \frac{10 \cdot ClockFrequency}{8 \cdot (UBRR + 1)} \end{equation}<br />
<br />
or<br />
<br />
\begin{equation}\label{eqnBytePeriod}\frac{1}{ByteRate} = \frac{10}{BitRate} = \frac{8 \cdot (UBRR + 1) \cdot 1000}{ClockFrequency} \,\, ms/byte \end{equation}<br />
<div>
<br /></div>
For the Arduino, \(ClockFrequency = 16 MHz \), such that \(9600\,bits/s\) is actually \(9615.4\,bits/s\) \((UBRR = 16,\,1.0406\,ms/byte)\), and \(115200\,bits/s\) is actually \(117647\,bits/s\,(UBRR = 207,\,85.144\,\mu{s}/byte)\).<br />
<br />
<a href="http://wormfood.net/avrbaudcalc.php?postbitrate=9600&postclock=16&u2xmode=1&ubrr8bit=1&hidetables=1" target="_blank">I found this nice AVR bit rate calculator</a>, if you are curious, you can play with it.<br />
<br />
<h2>
The Data</h2>
<div>
<br /></div>
Each graph consists on a log-log plot of two sets of data. The red curves are for the bit rate of 9600, and the blue curves are for the bit rate of 115200. Both curves refer to the transfer time of 32768 bytes. For each graph I have created an artificial byte overhead using a delay after receiving the byte. The log-log plot is necessary to linearize the "\(\frac{1}{x}\)" relation of total time versus block size.<br />
<br />
The tail of the curves can be estimated from equation \ref{eqDeltaTFinal} taking the limit when the block size is large:<br />
<br />
\begin{equation} \Delta t_{tail} = NumBytes \cdot \max \left( \frac{10}{BitRate}, O_{byte}\right) \end{equation}
<br />
<br />
which is \(\left(NumBytes \cdot \frac{10}{BitRate}\right)\) or \(\left(NumBytes \cdot O_{byte}\right)\). If we assume that \( O_{byte}\leqslant \frac{10}{BitRate}\), then for a sequence of 32768 bytes we have the theoretical values of 34.078 ms and 2.785 ms for 9615.4 bps and 117647 bps respectively. Which agrees whith the measured values of 34.1 ms and 2.79 ms.<br />
<br />
The graphs show that for the Arduino working in 9600 bps (9615.4 bps actually), a buffer size of 17 bytes is enough to reach the minimum theoretical transfer time. For 115200 bps (117647 bps actually) a buffer size of 27 bytes will do.<br />
<br />
give us the estimate of \(\max \left( \frac{10}{BitRate}, O_{byte}\right)\), while the linear part gives us the estimate of \(\frac{O_{block}}{BlockSize}\).<br />
<br />
Reality is always full of surprises. The tail behaves as we would expect from our crude model, but for the lower values of the buffer size, we can see some unexpected things.<br />
<br />
<br />
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=1380978792&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=1774840649&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=1775177843&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=309316184&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=552106527&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=1586597703&format=interactive" width="600"></iframe>
</div>
<div>
<iframe frameborder="1" height="371" scrolling="no" seamless="" src="https://docs.google.com/spreadsheets/d/1NdtC3Vq3C_8jZGFWBK_2fOApL8VtdVPoROrTRHKlWEw/pubchart?oid=1728596764&format=interactive" width="600"></iframe>
</div>
<br />
<div>
<br />
<h2>
Conclusion</h2>
<br />
Depending on the byte processing overhead your algorithm has, we saw that a 63 byte buffer can have the same performance in 9600 bits/s or 115200 bits/s.<br />
<br />
The bumpy block overhead is something that I might analyse more carefully some day in the future.<br />
<br /></div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-37370057240270918422015-09-05T14:13:00.000-03:002018-01-25T13:48:30.147-02:00A XSVF Assembler/Disassembler in python<h2>
Introduction</h2>
<div>
<br /></div>
As a sequence of my last project, the <a href="http://eeandcs.blogspot.com.br/2015/08/jtagxsvf-library-for-arduino.html" target="_blank">JTAG/XSVF library for Arduino</a>, I felt I needed a XSVF assembler and disassembler, so that I could hack JTAG a little bit. I found that XSVF is very convenient, much more than SVF when you are dealing with a single component in the JTAG chain. I also found out that the XSVF files produced by programmers are very inefficient and full of unnecessary stuf. It would be great if I could write my own XSVF files.<br />
<br />
The problem is that XSVF is a binary format. At first I started editing in binary with a program called ghex, but this is far from confortable. It is easy to get lost and almost impossible to maintain. So I decided to write an assembler.<br />
<br />
<a href="https://github.com/mrjimenez/JTAG" target="_blank">The code is here on github</a>.<br />
<br />
<h2>
The story</h2>
<br />
In fact, the disassembler was practically ready. Having written the XSVF player in a modular way, the disassembler was just another instance of the same code. Instead of playing XSVF, I would disassemble XSVF. Piece of cake.<br />
<br />
The XSVF player was in C++ and even though I could resuse the code, I'd much rather rewrite it in python. Python is about two orders of magnitude more productive than C/C++, even with all the years I have coding in those languages.<br />
<br />
There was also an issue with one XSVF instruction, namely XSDRINC. This is a problematic instruction and has been obsoleted. Xilinx's IMPACT software does not generate it anymore. This instruction has not been implemented in the JTAG library, the reason beeing that to support it I would need to start executing the instruction before finishing decoding it, or use an unacceptable amount of memory for a microcontroller (Arduino). It can certainly be done, but was slightly against the original philosophy of the code. Maybe I'll do it later, but since it was an obsolete instruction that I would not be able to test, I decided to skip it in order to quickly have a working XSVF player.<br />
<br />
I have basically translated the C++ code into python and the disassembler was ready, really no big deal. That code could now even be reused to write a XSVF player in python :). Anyway, hacking JTAG was the original motivation, so while the disassembler was a necessary tool during debugging, now I needed the assembler to start writting my own code in a maintainable way. Since I was now using python, I decided to implement XSDRINC, because I had not the same memory limitations I had in the Arduino.<br />
<br />
And it turns out that using python was indeed a good choice for other reasons.<br />
<br />
<h2>
Pyparsing</h2>
<br />
The first problem you face when you write a compiler or interpreter for a language is scanning and parsing. Scanning is classifying the original input in a sequence of valid tokens. Parsing is the syntatical analysis that validates that sequence of tokens. You need to check if the sentences are properly constructed according to a certain grammar.<br />
<br />
It turns out that programming a scanner and a parser even for an extremelly simple language like XSVF is a nightmare of details. Also, on the documenting side, the code is very distant from the actual grammar definition. Lets call python to the rescue...<br />
<br />
<a href="http://pyparsing.wikispaces.com/" target="_blank">Pyparsing</a> is a fantastic module that makes it very easy to program a scanner/parser in python. With the added bonus that you can use the python syntax to define a grammar that looks like BNF (Bakus-Naur Form), so close that it makes it unnecessary to document it in a separate BNF doc.<br />
<br />
In about 2 hours, I was able to learn how to use pyparsing and write my working XSVF parser. The code examples are very good, and reading the library code resolved some subtle issues.<br />
<br />
I had to struggle a little with some of my original ideas of supporting bytes in hexadecimal and binary. Specifying these in the language grammar was not that obvious, there was a subtle "order of matching" problem I had not seen comming. I guess I had in my mind a much straighter separation between scanning and parsing, therefore I missed well defining the language tokens. Anyway, it payed back to adapt my mind to the pyparsing way.<br />
<br />
<h2>
The language</h2>
<div>
<br /></div>
<div>
Since I had the assembler and the disassembler, I wrote the following code to test every instruction:</div>
<div>
<br /></div>
<div>
<script src="https://gist.github.com/mrjimenez/f99e6de6be9d07d95507.js"></script></div>
<div>
As you can see, I tried not to clutter the syntax while keeping it readable. There is a nice old school assembly style command (the semi-colon) that comments everything until the end of the line. Byte sequences can be written in binary or hexadecimal, without the overhead of using a prefix like "0x". These sequences are usually very large because they are normally used in programming or boundary scan. Beeing able to mix hexadecimal, binary and comments is a good way to keep it readable.<br />
<br />
The final test was to run the sequence assembler -> disassembler -> assembler and compare the two assembled files. I did it with a SHA1 hash, and after some debugging, they compared ok for the test file.</div>
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">$ cat asm_disasm_test.sh<br />#! /bin/bash<br />./XSVFAssembler.py > test.xsvf<br />./xsvf -c disasm -n test.xsvf > test.xsvf.s<br />./xsvf -c asm test.xsvf.s > test2.xsvf<br />sha1sum test*.xsvf</span></blockquote>
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">$ ./asm_disasm_test.sh<br />b1fcb2845c934c622d9b8ffff857d08c9542c8b4 test2.xsvf<br />b1fcb2845c934c622d9b8ffff857d08c9542c8b4 test.xsvf</span></blockquote>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitybPJr8xDA5pIL61pK_nB0sXSz_6T3xaTv3k3VhXNho08AW70Kf7db_-dXA1ReT_d6GcyEO5v3vrwlGHexlq-7GZdwSiTb_0OuwxJitXtBTq5Jl7tqrD1NCa1_TsjpmYK7YrsmY9lhJ0z/s1600/xsvf-hex-dump-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitybPJr8xDA5pIL61pK_nB0sXSz_6T3xaTv3k3VhXNho08AW70Kf7db_-dXA1ReT_d6GcyEO5v3vrwlGHexlq-7GZdwSiTb_0OuwxJitXtBTq5Jl7tqrD1NCa1_TsjpmYK7YrsmY9lhJ0z/s320/xsvf-hex-dump-2.png" width="315" /></a></div>
<div>
<br />
<h2>
Conclusion</h2>
<br />
Now I have a fully working assembler/disassembler for XSVF. That opens some doors for JTAG hacking and boundary scan testing. Let's see where it leads. If you decide to try it, please share your comments.<br />
<br />
<h2>
References</h2>
<div>
<br /></div>
<ol>
<li><a href="https://github.com/mrjimenez/JTAG" target="_blank">JTAG/XSVF library for Arduino</a></li>
<li><a href="https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form" target="_blank">Bakus-Naur Form</a></li>
<li><a href="https://pyparsing.wikispaces.com/" target="_blank">Pyparsing web page</a></li>
</ol>
<div>
<br /></div>
</div>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01CZTLHGE&asins=B01CZTLHGE&linkId=8c1857ca60588eb18e0832e8180342f3&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01EWNUUUA&asins=B01EWNUUUA&linkId=e19ef6aaf4dabf1393d03cdf74e7a4d5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00RRAM7PM&asins=B00RRAM7PM&linkId=5790a31bad78a8b03db7b9bec95a2d27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01L0ZL8N6&asins=B01L0ZL8N6&linkId=a4bff4046c0cd1b378c4192698032515&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPROFU&asins=B00NBPROFU&linkId=e5c6b5fbf584035d7a8f9e9e18cf4e27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPRQXK&asins=B00NBPRQXK&linkId=eb85e4e24d180083a35b69b3512ca7d2&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B074N7T1WS&asins=B074N7T1WS&linkId=7faa9bcaefca4f81e657fded958a5bf5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B0755VKZB5&asins=B0755VKZB5&linkId=3840a612e8bfeb8336b1d16786c37d34&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com1tag:blogger.com,1999:blog-1034836236404925218.post-63972594538739594622015-08-18T19:43:00.002-03:002023-10-17T10:40:04.094-03:00A JTAG/XSVF Library for Arduino<h2 class="tr_bq">
Introduction</h2>
<div>
<br /></div>
I have recently felt the need to incorporate a JTAG port in a project to program a hardware that contained a CPLD. The idea was to both program it and perform some integrity tests on the board. I imagined something using pogo pins, to make it easier and quicker to test everything. I would also write the necessary test routines and generate some kind of report.<br />
<br />
With this objective in mind, I have decided to design an Arduino shield to do the job. The testing routines were not really a big deal. And I was sure I would find some JTAG library for Arduino ready to be used. That was not the case.<br />
<br />
There were some projects using Arduino to control a JTAG TAP (Test Access Port), but they were all incomplete. And I had no idea what was really JTAG. So I had to study a little bit to make things work for me.<br />
<br />
In the end, the challenge proved enlightening. There were some caveats, both from hardware and from software. I'll try to address them in this article.<br />
<br />
The library is hosted in <a href="https://github.com/mrjimenez/JTAG" target="_blank">here in github</a>. It is also in the new Arduino IDE library manager, so it should be easy to find it.<br />
<br />
<h2>
After all, what is JTAG?</h2>
<div>
<br /></div>
In the internet days, everything starts with a good look at Wikipedia. <a href="https://en.wikipedia.org/wiki/Joint_Test_Action_Group" target="_blank">This is the link to JTAG on Wikipedia.</a> To make a long story short, here is the quote for JTAG definition on Wikipedia:<br />
<blockquote class="tr_bq">
The Joint Test Action Group (JTAG) is an electronics industry association formed in 1985 for developing a method of verifying designs and testing printed circuit boards after manufacture. In 1990 the Institute of Electrical and Electronics Engineers codified the results of the effort in IEEE Standard 1149.1-1990, entitled Standard Test Access Port and Boundary-Scan Architecture.</blockquote>
What it means is that the name JTAG originally meant an electronics industry association. But after IEEE published the Standard 1149.1, which referred to a "Standard Test Access Port", this test access port (TAP) has become more or less a synonym for JTAG.<br />
<br />
The physical layer consists of 5 signals:<br />
<br />
<ul>
<li>\(\hbox{TCK}\) - Test Clock input</li>
<li>\(\hbox{TMS}\) - Test Mode Select</li>
<li>\(\hbox{TDI}\) - Test Data Input</li>
<li>\(\hbox{TDO}\) - Test Data Output</li>
<li>\(\overline{\hbox{TRST}}\) - Test Reset Input</li>
</ul>
<div>
The \(\overline{\hbox{TRST}}\) signal is optional.<br />
<br />
JTAG enabled devices can be all connected together. The signals TCK and TMS (and \(\overline{\mbox{TRST}}\), if present) should be connected to all devices. As a consequence, all state machines in all devices will be always at the same state. The TDI of the JTAG interface is connected to the TDI of the first device. The TDO of a device should be connected to the TDI of the next device. Finally, the TDO of the last device is connected to the JTAG interface. This way, data can be shifted in or out of a big shift register, that gets bigger if you add more devices.</div>
<br />
<br />
The IEEE standard defines not only what the electric cable signals are, but also defines the logic state machine that must understand these signals. Objectives were to keep the number of signals on the cable as low as possible, but keeping the architecture versatile enough to perform any task.<br />
<br />
The standard goes on to define the basics of the Boundary-Scan Architecture, which seems to be what they had primarily in mind at that time. It is a way to of testing your hardware "on the fly", i.e., while the circuit is operating. Boundary-Scan compatible devices make it possible to control the logical value of output and input pins of your integrated circuits to see how the whole hardware will respond to those stimulus.<br />
<br />
<h2>
What is JTAG used for?</h2>
<div>
<br /></div>
It didn't take much to start "abusing" JTAG to perform other things besides boundary-scan. For example:<br />
<br />
<ul>
<li>Customized hardware testing for quality control</li>
<li>CPLDs and FPGAs programming</li>
<li>Microcontroller debugging</li>
</ul>
<div>
In order to understand how these tasks can be acheived, we must understand how the JTAG TAP works.<br />
<br /></div>
<h2>
The TAP</h2>
<div>
<br /></div>
<div>
The TAP is a synchronous finite state machine. The following diagram shows the state diagram of a JTAG TAP. Transitions are controlled by the state of TMS on the rising edge of TCK.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/JTAG_TAP_Controller_State_Diagram.svg/703px-JTAG_TAP_Controller_State_Diagram.svg.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="273" src="https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/JTAG_TAP_Controller_State_Diagram.svg/703px-JTAG_TAP_Controller_State_Diagram.svg.png" width="320" /></a></div>
<br />
<div>
The state machine is actually simple. In what follows, xR means DR or IR.</div>
<div>
<ul>
<li>Actions on the test logic can happen either on the risign or on the falling edge of TCK.</li>
<li>At any time, there are two registers that you have to be concerned with: the Instruction Register (IR) and the Data Register (DR). The actual content of DR depends on the value loaded on IR. IR must have at least two bits.</li>
<li>There is a reset state that is easily reacheable. Starting from any state in this diagram, if you hold TMS high for 5 consecutive clock rising edges, the TAP is guaranteed to enter TEST-LOGIC-RESET. When this state is entered, the IR is loaded with either the IDCODE instruction or the BYPASS instruction, so that when the TAP moves into RUN-TEST/IDLE no action will occur. If the signal TRST is present, it assynchronously forces the TAP into TEST-LOGIC-RESET.</li>
<li>RUN-TEST/IDLE is the sate where you will usually wait or pass between operations. It is one of the stable states, meaning if you hold TMS on a determined value, you stay in that state. For example, the instruction RUBINST causes a self-test of the system on this state.</li>
<li>The two vertical state columns perform similar funtions. The first is meant to read and write to the DR while the second does the same to the IR. The DR is actually a group of registers. The one you actually access is dependent on the contents of the IR, so you can think of IR as a selector.</li>
<li>SELECT-xR-SCAN: Temporary state to enter xR operations or just to pass on to the next state.</li>
<li>CAPTURE-xR: The current selected DR is loaded on the shift registers. In the case of the IR, this state loads the fixed binary pattern "01" on the bits closer to TDO. The other bits may load other design-specific data.</li>
<li>SHIFT-xR: In this state, the data loaded in the shift-register is both serially shifted in xR through TDI and shifted out through TDO.</li>
<li>EXIT1-xR: This is temporary state that provides a way to bypass PAUSE-xR and EXIT2-xR and go straight to UPDATE-xR.</li>
<li>PAUSE-xR: Nothing happens, the controller is paused while in this state.</li>
<li>EXIT2-xR: A temporary state that provides a way to return to SHIFT-xR.</li>
<li>UPDATE-xR: Data is latched into xR on the falling edge of TCK.</li>
</ul>
<div>
In other words, we write to IR to tell the device what we want, and write or read to DR to set a property or get a response.</div>
</div>
<div>
<br />
BYPASS is an instruction that turns DR into a one bit register that always capture a zero.<br />
<br /></div>
<div>
There is no such thing as shifting-in without shifting-out or vice-versa. We always do both.<br />
<br /></div>
<h2>
Playing around with JTAG</h2>
<div>
<br /></div>
<div>
The Inpact software from Xilinx has a nice interface that allows you to play with the TAP:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCpijsR40QDB2GValDgthIjj7T5vqvDn7YLP-erm4CxH0mCILAHjLDqGAIZDa_2-33YtmPLwkFq1dxvGQ85LnKTVjIiY82WnDFBATYgBxAXV2pANHXPpN-WyGWg4CX4MEmyFFlaFbhfoBU/s1600/impact1.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="194" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCpijsR40QDB2GValDgthIjj7T5vqvDn7YLP-erm4CxH0mCILAHjLDqGAIZDa_2-33YtmPLwkFq1dxvGQ85LnKTVjIiY82WnDFBATYgBxAXV2pANHXPpN-WyGWg4CX4MEmyFFlaFbhfoBU/s320/impact1.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
Select "Boundary-Scan", then "Initialize Chain" and then on the Debug menu choose "Enable/Disable Debug Chain". It is rudimentary and not very practical, but has its uses. Hand collecting of the bits shifted in and out of IR and DR can be tedious. That is why a second interface is provided that is slightly more usefull.</div>
<div>
<br /></div>
<div>
For example, on the previous screen, click on "Test Logic Reset", and this state becomes green. If the IEEE standard is folowed by this device, then the "IDCODE" instruction (if present) must have been loaded into IR. If we shift out DR, then we must be able to access this IDCODE. To do that, fill the box next to SCAN DR with 32 zeroes, click on "Execute" and lets see what comes out:</div>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">TDO Capture Data: 00000110111001011110000010010011</span></blockquote>
If we break this binary string into hexadecimal, we get something more meaningfull. This is 06E5E093, which is the IDCODE for XC2C64A.<br />
<br />
At this point, I should mention a very usefull file to have in hand. It is the BSDL file for the device. BSDL stands for "Boundary-Scan Definition Language", its a subset of VHDL that defines the boundary-scan parameters of a device. BSDL has been defined on the same IEEE Std 1149.1. There we can look at the instructions that our device will accept. The relevant part for us now is:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">...</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;"> attribute INSTRUCTION_LENGTH of xc2c64a : entity is 8;<br />attribute INSTRUCTION_OPCODE of xc2c64a : entity is<br /> "INTEST (00000010)," &<br /> "BYPASS (11111111)," &<br /> "SAMPLE (00000011)," &<br /> "EXTEST (00000000)," &<br /> "IDCODE (00000001)," &<br /> "USERCODE (11111101)," &<br /> "HIGHZ (11111100)," &<br /> "ISC_ENABLE_CLAMP (11101001)," &<br /> "ISC_ENABLEOTF (11100100)," &<br /> "ISC_ENABLE (11101000)," &<br /> "ISC_SRAM_READ (11100111)," &<br /> "ISC_SRAM_WRITE (11100110)," &<br /> "ISC_ERASE (11101101)," &<br /> "ISC_PROGRAM (11101010)," &<br /> "ISC_READ (11101110)," &<br /> "ISC_INIT (11110000)," &<br /> "ISC_DISABLE (11000000)," &<br /> "TEST_ENABLE (00010001)," &<br /> "BULKPROG (00010010)," &<br /> "ERASE_ALL (00010100)," &<br /> "MVERIFY (00010011)," &<br /> "TEST_DISABLE (00010101)," &<br /> -- "STCTEST (00010110)," &<br /> "ISC_NOOP (11100000)";</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">attribute INSTRUCTION_CAPTURE of xc2c64a : entity is </span><span style="font-family: "courier new" , "courier" , monospace;">"XXXXXX01" ;</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">attribute IDCODE_REGISTER of xc2c64a : entity is "XXXX0110111001011XXX000010010011";</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">attribute USERCODE_REGISTER of xc2c64a : entity is "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">...</span></blockquote>
That tells us that the IR has 8 bits and what codes it does understand. Lets try IDCODE again, now the hard way. Put "0000001" in the "Scan IR" box, then click "Execute". Then, if it is not already there, put "00000000000000000000000000000000" in the "Scan DR" box and click the second "Execute" button. You should get<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">TDO Capture Data: 00000101<br />TDO Capture Data: 00000110111001011110000010010011</span></blockquote>
The first line with 8 bits seem to be the mandatory "01" that IR will always load, plus six bits that have a meaning set by the manufacturer. This is what is implied by the line 'attribute INSTRUCTION_CAPTURE of xc2c64a : entity is "XXXXXX01" ;' Notice that since we must enter "CAPTURE-IR" before "UPDATE-IR", there is no way for the device to guess what instruction is going to be loaded. So this value is either a constant or dependent upon an internal state of the device. We will see later that it has something to do with checking for read/write protect.<br />
<br />
The second line with 32 bits is exactly what we got before when we performed the "Shift-DR" straight after "TEST-LOGIC-RESET".<br />
<br />
It is possible to use Impact to check that we have really got the right id code, if you go out of "Debug Mode", click on the device and then on "Available Operations" click "Get Device ID":<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">INFO:iMPACT - Current time: 8/14/15 5:01 PM<br />Maximum TCK operating frequency for this device chain: 33000000.<br />Validating chain...<br />Boundary-scan chain validated successfully.<br />'1': IDCODE is '00000110111001011110000010010011'<br />'1': IDCODE is '06e5e093' (in hex).<br />'1': : Manufacturer's ID = Xilinx xc2c64a, Version : 0</span></blockquote>
<div>
It is possible to crak a little bit more the of the id code. Bits 31 to 28 "0000" are the version number, bits 27 to 22 (6E5E in hexa) are the part number and the final 11 bits "093" (in hexa) are the manufacturer's id and in this case mean Xilinx. The first bit, bit 0 is in fact required to be '1' if an IDCODE instruction is present. Remember that upon TEST-LOGIC-RESET, IR gets loaded with IDCODE or BYPASS. But BYPASS is required to load a '0' at the start of the scan cycle, so this bit is used to identify which instruction has been used upon reset.<br />
<br /></div>
<h2>
Playing around with SVF</h2>
<div>
<br /></div>
<div>
Now we know enough about JTAG, lets see what else we can do. Suppose you want to describe a set of operations to be performed on a JTAG TAP controller, for example, suppose you want to program a certain device. How do you describe what has to be done? The answer is a programming language. SVF stands for "Serial Vector Format", and is a file format that specifies how and which boundary-scan vectors should be transferred to a device, and also which should be some of the expected results, like ID code or checksum.</div>
<div>
<br /></div>
<div>
The problem with SVF is that it is too verbose. Good for humans to read, but excessive for computers to deal with. So Xilinx has come up with XSVF, which is a binary form o SVF.</div>
<div>
<br />
Fortunately for us, Impact is able to generate both SVF and XSVF. In our previous example, go to menu "Output -> SVF File -> Create SVF File..." and choose a name for the file to save data in. Double click on "Get Device ID" then go to "Output -> SVF File -> Stop Writing to SVF File".<br />
<br />
You should get a file like this:<br />
<br /></div>
<div>
<script src="https://gist.github.com/mrjimenez/0a13b945688813ef3d7b.js"></script>
</div>
<div>
There is a quick reference for SVF and XSVF in the apendices of the document XAPP503 - SVF and XSVF File Formats for Xilinx Devices (<a href="http://www.xilinx.com/support/documentation/application_notes/xapp503.pdf">http://www.xilinx.com/support/documentation/application_notes/xapp503.pdf</a>).<br />
<br />
The most relevant information is in the instructions SIR and SDR. These are a version of the Impact gui interface that we have previously used. But besides telling the SVF machine what it should shift in, it also tell it what it should get shifted out. It also specifies masks, so that the relevant bits may be checked. For example:<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">SIR 8 TDI (01) SMASK (ff) ;<br />SDR 32 TDI (00000000) SMASK (ffffffff) TDO (f6e5f093) MASK (0fff8fff) ;</span></blockquote>
This means "shift the following 8 bits into IR: 00000001, all of which are relevant, then shift 32 zeros into DR, all of which are relevant, then compare the value received with f6e5f093, after masking both with 0fff8fff". Sounds familiar? Pretty much what we did by hand before, except the masking part.<br />
<br />
The comments in the file are interesting. And we can see a lot of redundant operations. The ID code is checked three times, and the read/write protect is checked twice. In the end, the device is put on BYPASS.<br />
<br /></div>
<div>
<h2>
Playing around with XSVF</h2>
<div>
<br /></div>
If we use Impact to generate a XSVF version of the ID code check program, we get this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitg4YNfPh_7hqz9Clxzvklxm9zPmgMfC7681FinXKOQA0L7AH5oS0ORyqyo9rVT4R2hpLA0flIYNWu-DtzeiXSBaxpNfdoSASd8bAxc_pj8MuQ2KgSh4ejj8rlybXQFJcduGFOEknPAvXC/s1600/xsvf-hex-dump.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitg4YNfPh_7hqz9Clxzvklxm9zPmgMfC7681FinXKOQA0L7AH5oS0ORyqyo9rVT4R2hpLA0flIYNWu-DtzeiXSBaxpNfdoSASd8bAxc_pj8MuQ2KgSh4ejj8rlybXQFJcduGFOEknPAvXC/s320/xsvf-hex-dump.png" width="320" /></a></div>
<br />
A completely binary file, improper for humans to read but good enough for computers. There you have the exact same program as before, but coded in binary XSVF instructions, rather than SVF. I wrote a disassembler in python for debugging, and the output is the following:<br />
<br />
<div>
<script src="https://gist.github.com/mrjimenez/b35e9b8a52b972464657.js"></script></div>
<br />
We can see the same redundancies that were present in the SVF file.<br />
<br />
The previously mentioned XAPP503 has the documentation for the XSVF instructions. Which boils down to shifting stuff into IR or DR, reading stuff back and pulsing TCK while in some state, usually in . Not new stuff, after all we have been through.<br />
<br />
<h2>
Arduino, at last</h2>
<div>
<br /></div>
Now that we know what JTAG is, we can program a microprocessor to do the job for us. Arduino is a nice choice for a number of reasons I don't need to get into. I have searched for a library that would just do what I wanted, but none seemed to work with my hardware. There were several problems, I'll try to address them.<br />
<br />
The first issue I found was memory usage. It is absolutely essential that you keep your strings within flash. Even more when you are in debug mode, where you want lots of output to understand what is going on.<br />
<br />
The second problem was the Arduino serial interface, which has no flow control. That means that we must provide one. What I did was to always use a fixed block transfer size. I have used the Arduino software serial buffer avoiding unecessary copying to spare memory. There are 64 bytes in this buffer, but only 63 are useable because of the circular buffer implementation. I have managed to increase its size to 256 bytes, the process is documented in the code. I have spent quite some time to make the transfer as fast as possible.<br />
<br />
The cable was another issue. Don't make it too long and do some termination/impedance matching. Reflections in TCK will kill any attempt to program a JTAG device. I have used three voltage dividers to convert from the 5 Volts logic to the 3.3 Volts logic of the devices I was using, and I have chosen the resistor values to kinda match the expected impedance without killing Arduino's ATMEGA 328p output drivers.<br />
<br />
The best solution for the signal conversion would have been to use buffers. Usually the JTAG cable will have a VCC signal that can be used to sense the device's operating voltage. That does not exclude the impedance matching, that in this case could be done with some series resistors.<br />
<br />
The VCC signal of the JTAG cable is used in this project to detect that the cable is actually connected to some hardware.<br />
<br />
Here is a picture of the two programmers I have used, the Xilinx one and the Arduino. Each of them is connected to a XC2C64A breakout board from Dangerous Prototypes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgToG8lCJ3RUNIgt13kibBVxGgmGMGzZq42Scygy0O7D4gkUS0-6KXW7zH5M7cYgiBevlbI6bTm8oJvONz_44AEYXE-vR7VK8PGoF0uYevlQSuLEFrZj3HWUXvPqAbFalzviGJpqVLtdCrl/s1600/arduino_jtag.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgToG8lCJ3RUNIgt13kibBVxGgmGMGzZq42Scygy0O7D4gkUS0-6KXW7zH5M7cYgiBevlbI6bTm8oJvONz_44AEYXE-vR7VK8PGoF0uYevlQSuLEFrZj3HWUXvPqAbFalzviGJpqVLtdCrl/s320/arduino_jtag.jpg" width="320" /></a></div>
<br />
Here it is a copy/paste of the terminal screen while programming an example:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">$ ./xsvf ../xsvf/XC2C64A/VHDL-CPLDIntro3LEDinverse.xsvf<br />File: /home/user/sketchbook/arduino/libraries/JTAG/extras/xsvf/XC2C64A/VHDL-CPLDIntro3LEDinverse.xsvf<br />Ready to send 22846 bytes.<br />IMPORTANT: Free memory: 771 bytes.<br />Sent: 22846 bytes, 0 remaining ()<br />IMPORTANT: XCOMPLETE<br />IMPORTANT: ********<br />IMPORTANT: Success!<br />IMPORTANT: ********<br />IMPORTANT: Processed 1417 instructions.<br />IMPORTANT: Checksum: 0x36/22846.<br />IMPORTANT: Sum: 0x0033D4CA/22846.<br />Received device quit: Exiting!<br /> Expected checksum: 0x36/22846.<br /> Expected sum: 0x0033D4CA/22846.<br />Elapsed time: 4.31 seconds.</span></blockquote>
<div>
<br /></div>
<h2>
Now what?</h2>
Besides programming devices, it is now possible to use JTAG to actually communicate with devices. One possibility is to create a JTAG TAP in VHDL and use it to control your device.<br />
<br />
Another possibility is to use the Arduino JTAG to hack into hardware. It is not unusual to find "lost" and undocumented JTAG interfaces on several devices, if you search a little around the internet, you will have some ideas of what you can do.<br />
<br />
Hope you like this, happy jtagging!<br />
<br />
<h2>
Editions</h2>
<br />
<ol>
<li>There is a nice tutorial on YouTube int the channel "Electronica y Linux": <a href="https://www.youtube.com/watch?v=x5eW9H1GoeA">https://www.youtube.com/watch?v=x5eW9H1GoeA</a></li>
</ol>
<br />
<br />
<br /></div>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01CZTLHGE&asins=B01CZTLHGE&linkId=8c1857ca60588eb18e0832e8180342f3&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01EWNUUUA&asins=B01EWNUUUA&linkId=e19ef6aaf4dabf1393d03cdf74e7a4d5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00RRAM7PM&asins=B00RRAM7PM&linkId=5790a31bad78a8b03db7b9bec95a2d27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01L0ZL8N6&asins=B01L0ZL8N6&linkId=a4bff4046c0cd1b378c4192698032515&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPROFU&asins=B00NBPROFU&linkId=e5c6b5fbf584035d7a8f9e9e18cf4e27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B00NBPRQXK&asins=B00NBPRQXK&linkId=eb85e4e24d180083a35b69b3512ca7d2&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B074N7T1WS&asins=B074N7T1WS&linkId=7faa9bcaefca4f81e657fded958a5bf5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B0755VKZB5&asins=B0755VKZB5&linkId=3840a612e8bfeb8336b1d16786c37d34&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com18tag:blogger.com,1999:blog-1034836236404925218.post-90483652409413677342014-10-19T19:56:00.000-02:002015-08-18T19:47:48.308-03:00How to post code on Blogger using Gist<h3>
The problem</h3>
How to post nice pieces of code to a blog? The desired properties are:<br />
<ol>
<li>Beauty.</li>
<li>Language awareness.</li>
<li>Easy to copy to the clipboard.</li>
<li>Easy to integrate in Blogger.</li>
<li>Easy to maintain</li>
</ol>
<h3>
One possible solution</h3>
Use <a href="https://gist.github.com/">Gist</a>.You must have an account on GitHub. Gists are versioned text files, so you can alter them and fix any problems on the Gist page without having to deal with your blog page.<br />
<br />
<h3>
How to integrate it into Blogger</h3>
After creating your Gist, copy the "Embed URL" to the clipboard:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLs2S8nJf7Gp6KyzaDM_58nMe9ZehGTGwz6vSRpMTNzcLOMqUNNrd2PW2XhWkeOifNw-wZ18zHIUf9IsxGiJBKG1LXqcXfk6AHR216nuoOYDy28zLMwFsMTfEbgISGHT-7hFPTjOmi38Id/s1600/gist-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLs2S8nJf7Gp6KyzaDM_58nMe9ZehGTGwz6vSRpMTNzcLOMqUNNrd2PW2XhWkeOifNw-wZ18zHIUf9IsxGiJBKG1LXqcXfk6AHR216nuoOYDy28zLMwFsMTfEbgISGHT-7hFPTjOmi38Id/s1600/gist-1.png" /></a></div>
The link is something like this:<br />
<blockquote class="tr_bq">
<b><script src="https://gist.github.com/mrjimenez/bceea54fdf66a9aec7b2.js"></script></b></blockquote>
Reserve a space to integrate the Gist in your blog post. Then choose "HTML" in the Compose/HTML radio button. Insert that code in the proper place in your page, and your're done:<br />
<br />
<script src="https://gist.github.com/mrjimenez/bceea54fdf66a9aec7b2.js"></script>
<br />
Take a look at the snapshot:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvrbwdPeJEilG9AgI7dImiMbc9BwSsaOfHK4Phur1X2tg6QQS-sNPDUrrGG1blzaj3G2pdhdwfTXAbHSItKAT2zu7jbWOdha7uta74DeRnu-WCvXChk1XJwnedjFlLQ5CqKwmU7jH4SXHh/s1600/gist-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvrbwdPeJEilG9AgI7dImiMbc9BwSsaOfHK4Phur1X2tg6QQS-sNPDUrrGG1blzaj3G2pdhdwfTXAbHSItKAT2zu7jbWOdha7uta74DeRnu-WCvXChk1XJwnedjFlLQ5CqKwmU7jH4SXHh/s1600/gist-2.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-42038617131996797122014-10-18T11:35:00.002-03:002014-10-18T11:35:45.185-03:00Uploading photos to Facebook using google-chrome on linux<h3>
The problem</h3>
Uploading photos to Facebook seems to be unresponsive, I get to this screen and when I press the "Choose File" button, nothing happens:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiru3u6eYFKouKkLTvYBg9ZRFRHLtLQndAgZIWe4R8-aeIMBzfmAgiXazF0UL3RiVW92sNurB496per3Umdeh63wDnrOwBG8_p304IiVqgxpVFBPDhziRUtDs5JAwVedGS29cE0__gmsC9-/s1600/facebook-upload-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiru3u6eYFKouKkLTvYBg9ZRFRHLtLQndAgZIWe4R8-aeIMBzfmAgiXazF0UL3RiVW92sNurB496per3Umdeh63wDnrOwBG8_p304IiVqgxpVFBPDhziRUtDs5JAwVedGS29cE0__gmsC9-/s1600/facebook-upload-1.png" height="200" width="320" /></a></div>
<br />
<h3>
The solution</h3>
Go to a file manager (e.g, konqueror) drag the file with the mouse and drop it over one of the buttons. The file will then be accepted:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQw0bv0moGhD1DkG5DxCommmw3xTrZ3Syzwkx_f-HDoU77xi7bsu9fDZ7QUiocSE024_jKjug-thRFvidD4wSXwhS_UCEKCLUJF_QQItnuAwnDe7cKptCVdAReDhB9mOVJj0yRD-m4RwvK/s1600/facebook-upload-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQw0bv0moGhD1DkG5DxCommmw3xTrZ3Syzwkx_f-HDoU77xi7bsu9fDZ7QUiocSE024_jKjug-thRFvidD4wSXwhS_UCEKCLUJF_QQItnuAwnDe7cKptCVdAReDhB9mOVJj0yRD-m4RwvK/s1600/facebook-upload-2.png" height="195" width="320" /></a></div>
<br />
If you want to unselect a photo, click the corresponding "Choose File" button. Apparently, its intended function is not to choose a file, it is to reset the file to "No file choosen":<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMjvEs4kpxwLaWSxMl9pGi_FrJjLviAkaZ4dheypTGpLnpchk2tKZ-Pj9uCHodOOKbWx8oggHobMK0my2gnuMawvecTd7vPtfasbP5RtrNo7rWxiBDvG3CK2QTtfMxXCP_X0dXy7ZUZ9d2/s1600/facebook-upload-3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMjvEs4kpxwLaWSxMl9pGi_FrJjLviAkaZ4dheypTGpLnpchk2tKZ-Pj9uCHodOOKbWx8oggHobMK0my2gnuMawvecTd7vPtfasbP5RtrNo7rWxiBDvG3CK2QTtfMxXCP_X0dXy7ZUZ9d2/s1600/facebook-upload-3.png" height="199" width="320" /></a></div>
<br />
After that, click "Upload Photos" and wait, because it takes some time.<br />
<br />Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-74650216917526675912014-10-16T22:41:00.000-03:002014-10-17T07:11:18.113-03:00Cuda 6.5 on OpenSuSE 12.3<h3>
NVidia Drivers</h3>
Make sure you have the official NVidia drivers installed in your system:<br />
<div>
<ul>
<li>Run Yast.</li>
<li>Click in "Software Repositories"</li>
<li>Click in "Add"</li>
<li>Choose "Specify URL", then "Next"</li>
<li>Repository Name: "nvidia", URL: http://download.nvidia.com/opensuse/12.3/</li>
<li>Confirm</li>
<li>Go back to Yast</li>
<li>Click in "Software Management"</li>
<li>Search for "nvidia"</li>
<li>Add the following packages: </li>
</ul>
<ol>
<li>x11-video-nvidiaG03-340.46-30.1.x86_64</li>
<li>nvidia-gfxG03-kmp-desktop-340.46_k3.7.10_1.1-30.1.x86_64</li>
<li>nvidia-computeG03-340.46-30.1.x86_64</li>
<li>nvidia-settings-325.15-1.3.x86_64</li>
<li>nvidia-glG03-340.46-30.1.x86_64</li>
<li>nvidia-texture-tools-2.0.6-36.2.x86_64</li>
<li>nvidia-uvm-gfxG03-kmp-desktop-340.46_k3.7.10_1.1-30.1.x86_64</li>
</ol>
Notice that the above assumes your board is supported by the G03 kernel driver and that you are using the "kernel-desktop". Make sure you choose the proper driver for your board and the kernel driver corresponding to your kernel.<br />
<br /></div>
<h3>
CUDA Installation</h3>
<div>
Install the CUDA repository. Although the repository is for OpenSuSE 13.1, it will work perfectly with 12.3.</div>
<div>
<ul>
<li>Click in "Add"</li>
<li>Choose "Specify URL", then "Next"</li>
<li>Repository Name: "cuda", URL: http://developer.download.nvidia.com/compute/cuda/repos/opensuse131/x86_64</li>
<li>Confirm</li>
<li>Go back to Yast</li>
<li>Click in "Software Management"</li>
<li>Search for "cuda"</li>
<li>Add the following packages (some of them will be automatically added): </li>
</ul>
</div>
<div>
<ol>
<li>cuda-documentation-6-5-6.5-14.x86_64</li>
<li>cuda-cudart-6-5-6.5-14.x86_64</li>
<li>cuda-cufft-dev-6-5-6.5-14.x86_64</li>
<li>cuda-repo-opensuse131-6.5-14.x86_64</li>
<li>cuda-visual-tools-6-5-6.5-14.x86_64</li>
<li>cuda-cufft-6-5-6.5-14.x86_64</li>
<li>cuda-npp-dev-6-5-6.5-14.x86_64</li>
<li>cuda-curand-dev-6-5-6.5-14.x86_64</li>
<li>cuda-license-6-5-6.5-14.x86_64</li>
<li>cuda-runtime-6-5-6.5-14.x86_64</li>
<li>cuda-misc-headers-6-5-6.5-14.x86_64</li>
<li>cuda-samples-6-5-6.5-14.x86_64</li>
<li>cuda-curand-6-5-6.5-14.x86_64</li>
<li>cuda-toolkit-6-5-6.5-14.x86_64</li>
<li>cuda-cublas-6-5-6.5-14.x86_64</li>
<li>cuda-cusparse-dev-6-5-6.5-14.x86_64</li>
<li>cuda-drivers-340.29-0.x86_64</li>
<li>cuda-cudart-dev-6-5-6.5-14.x86_64</li>
<li>cuda-npp-6-5-6.5-14.x86_64</li>
<li>cuda-command-line-tools-6-5-6.5-14.x86_64</li>
<li>cuda-cusparse-6-5-6.5-14.x86_64</li>
<li>cuda-6.5-14.x86_64</li>
<li>cuda-core-6-5-6.5-14.x86_64</li>
<li>cuda-cublas-dev-6-5-6.5-14.x86_64</li>
<li>cuda-driver-dev-6-5-6.5-14.x86_64</li>
<li>cuda-6-5-6.5-14.x86_64</li>
</ol>
<ul>
<li>Click "Accept".</li>
</ul>
<br />
<ul>
</ul>
<h3>
Testing</h3>
</div>
<div>
<ul>
<li><a href="http://sourceforge.net/projects/cuda-z/files/cuda-z/0.8/CUDA-Z-0.8.207.run/download" target="_blank">Download CUDA-Z here</a></li>
<li>Run CUDA-Z as root. Here is what I get:</li>
</ul>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7SHY2Y1zJN8i0NIFZeGWVD4OJMRWVhsCugybQigxPsZ3koQoIBDsun78b8KgwHGTIRk5G7MwaI_IwcTdQ2iqVt-f35M8aKMCtJJcvec7J4FUUkDK5W9kIJozu7ffw9_qBt8DMVZoqreNT/s1600/cuda-z-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7SHY2Y1zJN8i0NIFZeGWVD4OJMRWVhsCugybQigxPsZ3koQoIBDsun78b8KgwHGTIRk5G7MwaI_IwcTdQ2iqVt-f35M8aKMCtJJcvec7J4FUUkDK5W9kIJozu7ffw9_qBt8DMVZoqreNT/s1600/cuda-z-1.png" height="320" width="256" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixUlH5hsvU2DFVFLbtSkCZvofDVzwvSIMt0zJtFGjcBI6hz3Ufp7M61hb-xrsRhZp1qOkoOZGt5a5J9iTHaN0-k6cratvt3wGtIhNACkAZIr8xSNuQ5sC93amCPSmlQGQCi87M03fRTPK2/s1600/cuda-z-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixUlH5hsvU2DFVFLbtSkCZvofDVzwvSIMt0zJtFGjcBI6hz3Ufp7M61hb-xrsRhZp1qOkoOZGt5a5J9iTHaN0-k6cratvt3wGtIhNACkAZIr8xSNuQ5sC93amCPSmlQGQCi87M03fRTPK2/s1600/cuda-z-2.png" height="320" width="256" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6290iXdhheodP7ZExaKeiVP_2Hx9YoxxTF_1Bh7qPOYdPc373EiHMeLhc6vup0gfbq2NjO5iwqBKTm2vsthKlSEGTi1BInAEF1SCMlySMAZEdxjoknthnddEcmSHRFVrXPTfaG9tvNG5B/s1600/cuda-z-3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6290iXdhheodP7ZExaKeiVP_2Hx9YoxxTF_1Bh7qPOYdPc373EiHMeLhc6vup0gfbq2NjO5iwqBKTm2vsthKlSEGTi1BInAEF1SCMlySMAZEdxjoknthnddEcmSHRFVrXPTfaG9tvNG5B/s1600/cuda-z-3.png" height="320" width="256" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWRNOFZ4CdQQmD-Pa-lkWUhO0wITQ6cKijMUwAUZtNmlmBivyrNwqflCY_ViUjcTx6h0qzE0Y22llIc4BYtsjJGj9wwb1T7fsBX9hFxSrI2i60QDqoIougI7bqwmf3NU968AaXRp5Fkktx/s1600/cuda-z-4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWRNOFZ4CdQQmD-Pa-lkWUhO0wITQ6cKijMUwAUZtNmlmBivyrNwqflCY_ViUjcTx6h0qzE0Y22llIc4BYtsjJGj9wwb1T7fsBX9hFxSrI2i60QDqoIougI7bqwmf3NU968AaXRp5Fkktx/s1600/cuda-z-4.png" height="320" width="256" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3>
Usefull Links</h3>
</div>
<div>
<ul>
<li><a href="http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux" target="_blank">NVidia Linux CUDA page</a></li>
</ul>
</div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com0tag:blogger.com,1999:blog-1034836236404925218.post-84849547572918310122014-10-06T23:13:00.003-03:002015-09-28T08:57:25.978-03:00Moving Averages Using Google Apps Scripts<h2>
Abstract</h2>
A moving average (MA) process, also known as weighted moving average (WMA) is a type of signal filtering that consists in performing a weighted average over a finite sequence of past samples of the original signal. Implementing such scheme in a worksheet is not always straightforward due to the handling of missing values. This article proposes an implementation of such filters on Google Sheets using Google App Scripts, a Javascript like language. Exponential Moving Average (EMA) filters are an important special case of WMA, so they have been implemented on top of WMA.<br />
<br />
<h2>
Keywords</h2>
Moving Average, MA, Weighted Moving Average, WMA, Exponential Moving Average, EMA, Noise Removal, Filtering, Google Sheets, Spreadsheets, Google App Scripts, Javascript.<br />
<br />
<h2>
Introduction</h2>
<div>
In previous articles [<a href="http://eeandcs.blogspot.com/2014/09/weighted-moving-averages-on-google.html" target="_blank">1</a>][<a href="http://eeandcs.blogspot.com/2014/09/exponential-moving-averages-on-google.html" target="_blank">2</a>], implementations of WMA and EMA have been proposed using the normal spreadsheet function infrastructure. These implementations try to be compatible with existing spreadsheet standard functions so that they can be easily ported to other spreadsheet e.g., OpenOffice.</div>
<div>
<br /></div>
<div>
The inconvenient in this solution is that the formulas are large and as a consequence, hard to read and maintain, making it easy to slip subtle errors. Also, the formula must be called once for each line to be calculated, and this process has a big overhead. The ideal solution should make a single call to a function that would return an array of processed data. The following implementation addresses both issues.</div>
<div>
<br /></div>
<h2>
Implementation</h2>
<div>
<script src="https://gist.github.com/mrjimenez/bceea54fdf66a9aec7b2.js"></script>
</div>
<h2>
Results</h2>
<div>
<a href="https://docs.google.com/spreadsheets/d/17VDGSihdRVeNGB8jQGw0iOhyVIQ5vMcFEFeM3v0fd98/edit?usp=sharing" target="_blank">The resulting spreadsheet</a> shows a comparison of the previously posted methods with this scripted based one.<br />
<br /></div>
<h2>
Conclusion</h2>
<div>
Two javascript functions have been developed to implement the missing data weighted moving averages of previous articles [<a href="http://eeandcs.blogspot.com/2014/09/weighted-moving-averages-on-google.html" target="_blank">1</a>][<a href="http://eeandcs.blogspot.com/2014/09/exponential-moving-averages-on-google.html" target="_blank">2</a>]. The results have been shown to be identical. The javascript based method has the advantage of being much cleaner to maintain, typically requiring a single cell on the spreadsheet.</div>
<div>
<br /></div>
<div>
<br /></div>
Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com5tag:blogger.com,1999:blog-1034836236404925218.post-51684637450441872982014-09-24T00:27:00.001-03:002015-09-28T08:58:20.761-03:00Weighted Moving Averages on Google Sheets<h2>
<span style="font-size: large;">Abstract</span></h2>
A moving average (MA) process, also called weighted moving average (WMA) is a type of signal filtering that consists in performing a weighted average over a finite past sequence of samples of the original signal. Implementing such scheme in a worksheet is not always straightforward due to the handling of missing values. This article proposes a convenient way to implement such filters on Google Sheets.<br />
<br />
<h2>
<span style="font-size: large;">Keywords</span></h2>
Weighted Moving Average, MA, WMA, Noise Removal, Filtering, Google Sheets, Spreadsheets.<br />
<br />
<h2>
<span style="font-size: large;">Introduction</span></h2>
The moving average signal is defined by:<br />
\[y(j) = \sum_{i=0}^n w(i) \cdot x(j-n+i)\]<br />
The spreadsheet function \(AVERAGE()\) performs this computation in the particular case where all \(w(i)\) are equal to one. In this case, it is very easy to deal with missing values, as the only thing you need to do is count the number of non-missing values and divide the sum by this count.<br />
<br />
When the \(w(i)\)'s are different numbers, it is not enough to count the number of non-missing values, one needs to perform the final normalization using the sum of the correspondent weight coefficients.<br />
<br />
<h2>
<span style="font-size: large;">Proposed Solution</span></h2>
The proposed solution is to use the function \(SUMPRODUCT()\) twice, first to get the weighted sum, and second to get the sum of the coefficients that have multiplied non-missing data.<br />
<br />
Assume that:<br />
<ul>
<li>The spreadsheet has a page called "Filter1".</li>
<li>The averaging coefficients are on column Filter1!A.</li>
<li>Cell Filter1!B1 has the formula "COUNT(A:A)", which will count the number of averaging coefficients.</li>
<li>The current cell is B22.</li>
</ul>
Then the claim is that the following formula will calculate the correct value of WMA:<br />
\[\begin{array}{l}<br />
=\\<br />
SUMPRODUCT( \\<br />
\qquad OFFSET(B22, -Filter1!$B$1 + 1, 0, Filter1!$B$1, 1), \\<br />
\qquad OFFSET(Filter1!$A$1, 0, 0, Filter1!$B$1, 1)) \\<br />
/ \\<br />
SUMPRODUCT(\\<br />
\qquad ARRAYFORMULA(\\<br />
\qquad \qquad N(ISNUMBER(\\<br />
\qquad \qquad \qquad OFFSET(B22, -Filter1!$B$1 + 1, 0, Filter1!$B$1, 1)))), \\<br />
\qquad OFFSET(Filter1!$A$1, 0, 0, Filter1!$B$1, 1))<br />
\end {array}\]<br />
The details of this expression are as follows:<br />
<ul>
<li>\(Filter1!$B$1\) is \(n\).</li>
<li>\(OFFSET(current\_cell,−n+1,0,n,1)\) is used to produce a range of n cells, of which the current cell is the last one.</li>
<li>\(OFFSET(Filter1!$A$1, 0, 0, Filter1!$B$1, 1)\) are the weighting coefficients.</li>
<li>\(ARRAYFORMULA(N(ISNUMBER(OFFSET(\cdots))))\) applies the function \(N()\) to the boolean result of \(ISNUMBER()\) for each cell in the current range, which will produce an array of zeroes where there is missing data and ones where there is data.</li>
</ul>
<div>
<br /></div>
<h2>
<span style="font-size: large;">Results</span></h2>
The formula has been tested against some weighting data. The <a href="https://docs.google.com/spreadsheets/d/1BM4vlCGi1gjpXHB9sKW9c7f4sc3MBanhtwYGV_Rr8ew/edit?usp=sharing" target="_blank">resulting spreadsheet</a> has a plot of the original data, along with the \(AVERAGE()\) data and WMA data for comparison.<br />
<br />
<h2>
<span style="font-size: large;">Conclusion</span></h2>
A spreadsheet formula for the correct calculation of a weighted moving average has been derived and successfully tested on Google Sheets. The proposed formula deals with missing values in a way similar to the \(AVERAGE()\) function, avoiding the distortions that would be caused either by using zero in place of the missing values or by packing the original series.Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com3tag:blogger.com,1999:blog-1034836236404925218.post-61658624005949839122014-09-23T00:07:00.000-03:002018-01-11T20:52:20.600-02:00Exponential Moving Averages on Google Sheets<h2>
<span style="font-size: large;">Abstract</span></h2>
Exponential moving averages (EMA) is a way to remove noise from data series. Unfortunately, useful straightforward spreadsheet support for EMA is absent. This article examines the problems involved and proposes a one line formula solution to add EMA to a spreadsheet.<br />
<br />
<h2>
<span style="font-size: large;">Keywords</span></h2>
Exponential Moving Average, EMA, Noise Removal, Filtering, Google Sheets, Spreadsheets.<br />
<br />
<h2>
<span style="font-size: large;">Introduction</span></h2>
Traditionally, implementing an exponential moving average has been done in spreadsheets using a recursive formula, i.e., an auto-regressive (AR) process or an infinite impulse response (IIR) filter. The formula is the following:<br />
\[y(n) = \alpha \cdot x(n) + (1 - \alpha) \cdot y(n-1)\]<br />
There are some problems with this approach:<br />
<ol>
<li>Strictly speaking, this is not a moving average. Moving average assumes a finite length sliding window under which data is being weighted. There is a growing window, not a sliding window.</li>
<li>The last item implies that the average takes into account every single data sample. The previous formula actually implements an IIR filter. This kind of averaging never forgets a value, although old values certainly get irrelevant over time. It would be nice to have control over the window where the exponential average takes place, especially when one intends to use a small window.</li>
<li>It does not deal properly with missing values. In a spreadsheet, it is common to have missing data values and the previous approach does not allow them. Converting missing values to zero would unacceptably distort the average value. If one insists in applying this formula to the series by packing the original series, the weights applied to the values in the averaging process will not reflect the actual distance in time that these samples might have.</li>
</ol>
The function AVERAGE(range) is able to deal with this problem quite simply because it uses the same average on every sample, so it is just a matter of dividing the sum by the number of non-blank entries. In a non-uniform averaging like EMA, we need to keep track of which weights were really applied or not due to missing values, and fix the normalizing factor accordingly.<br />
<br />
<h2>
<span style="font-size: large;">Proposed solution</span></h2>
The function SERIESSUM(a, n, m, x) is defined as<br />
\[SERIESSUM(a, n, m, x) = \sum_{i=0}^n x_i a^{n+m i}.\]<br />
The proposed solution is to use this function twice, first to calculate the weighted sum, and then a second time to calculate the sum of the weights where data is not missing.<br />
Assume that:<br />
<ul>
<li>Cell F1 contains the geometric progression ratio \((\alpha)\);</li>
<li>Cell F2 contains the window size \((n)\);</li>
<li>Column B contains the raw data;</li>
<li>The current cell is B22.</li>
</ul>
Then the claim is that the following formula will calculate the correct value of EMA:<br />
\[\begin{array}{l}<br />
=\\<br />
SERIESSUM($F$1, $F$2, -1, \\<br />
\qquad ARRAYFORMULA(N(OFFSET(B22, -$F$2 + 1, 0, $F$2, 1))))\\<br />
/ \\<br />
SERIESSUM($F$1, $F$2, -1, \\<br />
\qquad ARRAYFORMULA(\\<br />
\qquad \qquad N(ISNUMBER(OFFSET(B23, -$F$2 + 1, 0, $F$2, 1)))))<br />
\end{array}\]<br />
The details of this expression are as follows:<br />
<ul>
<li>\(OFFSET(current\_cell, -n+1, 0, n, 1)\) is used to produce a range of \(n\) cells, of which the current cell is the last one.</li>
<li>\(ARRAYFORMULA(N(OFFSET(\dotsc)))\) will apply the \(N()\) function to each element of the argument range to generate a new range with zero values in the missing data cells. Without this trick, \(SERIESSUM()\) would use non-missing values as if they were contiguous.</li>
<li>\(ARRAYFORMULA(N(ISNUMBER(OFFSET(\dotsc))))\) will generate a range composed of ones where data is not missing and zeros where data is missing.</li>
</ul>
<div>
<h2>
<span style="font-size: large;">Results</span></h2>
</div>
<div>
The formula has been tested against some weighting data. The <a href="https://docs.google.com/spreadsheets/d/1diQdF6CECy5ryTCle5mOpO9hMO7M7XJOgk8UyckseVo/edit?usp=sharing" target="_blank">resulting spreadsheet</a> has a plot of the original data, along with the \(AVERAGE()\) data and EMA data for comparison.</div>
<div>
<br /></div>
<div>
<h2>
<span style="font-size: large;">Conclusion</span></h2>
</div>
<div>
A spreadsheet formula for the correct calculation of an exponential moving average has been derived and successfully tested on Google Sheets. The proposed formula deals with missing values in a way similar to the \(AVERAGE()\) function, avoiding the distortions that would be caused either by using zero in place of the missing values or by packing the original series.</div>
<div>
<br /></div>
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon&region=US&placement=B01CZTLHGE&asins=B01CZTLHGE&linkId=8c1857ca60588eb18e0832e8180342f3&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff" style="height: 240px; width: 120px;">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B01EWNUUUA&asins=B01EWNUUUA&linkId=e19ef6aaf4dabf1393d03cdf74e7a4d5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B00RRAM7PM&asins=B00RRAM7PM&linkId=5790a31bad78a8b03db7b9bec95a2d27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B01L0ZL8N6&asins=B01L0ZL8N6&linkId=a4bff4046c0cd1b378c4192698032515&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B00NBPROFU&asins=B00NBPROFU&linkId=e5c6b5fbf584035d7a8f9e9e18cf4e27&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B00NBPRQXK&asins=B00NBPRQXK&linkId=eb85e4e24d180083a35b69b3512ca7d2&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B074N7T1WS&asins=B074N7T1WS&linkId=7faa9bcaefca4f81e657fded958a5bf5&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>
<iframe style="width:120px;height:240px;" marginwidth="0" marginheight="0" scrolling="no" frameborder="0" src="//ws-na.amazon-adsystem.com/widgets/q?ServiceVersion=20070822&OneJS=1&Operation=GetAdHtml&MarketPlace=US&source=ac&ref=tf_til&ad_type=product_link&tracking_id=mrj00-20&marketplace=amazon®ion=US&placement=B0755VKZB5&asins=B0755VKZB5&linkId=3840a612e8bfeb8336b1d16786c37d34&show_border=false&link_opens_in_new_window=false&price_color=333333&title_color=0066c0&bg_color=ffffff">
</iframe>Marcelo Roberto Jimenezhttp://www.blogger.com/profile/00559195914754568252noreply@blogger.com13