rake SPEC_OPTS=-b

References: http://rspec.rubyforge.org/rspec/1.1.12/classes/Spec/Rake/SpecTask.html

A.  you are on a firewalled network, from where you can’t access github (remote port:9418 is not allowed)

or

B. you want to access a securized git server from home, but your IP is not on the git server’s whitelist.

and, of course, you have ssh access on a server that can do this.

A. Make a ssh tunnel to the git server

Me –> ServerWithSSHAccess –> GitServer

The comand to use is:

ssh ServerWithSSHAccess -L 2000:GitServer:22 -N ,

where 2000 is the local port you will connect to, and 22 is the remote port used to connect to the git server (if the connection uses git:// instead of default ssh port, change this to 9418). Then, instead of:

git clone GitServer/my_repo,

you must use:

git clone localhost:2000/my_repo

B. Automatically use tunnel to connect to git server

Furthermore, if the repository is already cloned on your computer, or you want to use the same addresses, you can config your ssh client to automatically pick the tunneled connection when connecting to the git server. In ~/.ssh/config, add:

Host GitServer

HostName 127.0.0.1

Port 2000

so now you can access your git repo as you were directly connected to it:

git clone GitServer/my_repo
git pull
git push

etc

This article is the english version of “Conseils pour utiliser Ruby, RVM, Passenger, Rails, Bundler, … en développement”, originaly written in French. If you have a better translation, I’d be happy to use it (full or parts) and credit you. Introduction : Why this advice The dev team I’m working in is composed of 3 people who use Ruby. Besides me, who’s is spending the most part of my time on Ruby and became “the one who knows” about this stuff, m … Read More

via Jérémy Lecour

XML parsers

The most popular XML parsers can be split in 2 big categories:

• tree-based (aka DOM parsers) – that parse the whole XML file and transform it into a huge tree of nodes.
• event-based (SAX and Pull parsers)

Therefore, a DOM parser consumes a lot of memory (since it stores the whole tree in memory) but it is much easier to use. You can do all sorts of  cool stuff on them like xpath selectors, css selectors, converting the xml to a hash, etc. Basically, the vast majority of examples/tutorials of ruby XML are refering to DOM parsers.

However, in the 5% of the situations where the XML file is really big, storing it in memory is not an option.

Event-based XML parsers

There are two types of event-based parsers

• Push parsers like SAX, where you react to encoutered tags as you get them. You can find here a comparison over the principal ruby SAX parsers.

• Pull parsers, where you control a “cursor” in the XML file that you can move with simple primitives like go up/go down etc.

Declarative XML SAX parsing library

There is an interesting XML SAX parsing library that automatically parses the file into an object. However, the programmer must declare the structure of the subtree. Check out SaxMachine.

Example using Nokogiri’s Pull Parser

I wanted to implement a method that would iterate over <product> tags. This method would also transform the subtree <product> … </product> to a hash. I don’t have a standard subtree structure. Below is a snipped from my function.

      def each_offer(input, options={}, &block)
        reader = Nokogiri::XML::Reader input
        tag_name = options[:tag_name] || 'product'

        #search for reader
        until (reader.name == tag_name)
          break unless reader.read
        end

        i            = 1
        level        = 1
        product_node = nil
        elem_name    = nil
        elem_hash = {}

        stack = [[tag_name, reader.attributes]]
        while (reader.read)
          case reader.node_type
            #start element
            when 1
              stack.push([reader.name.to_s, reader.attributes])

            #text element
            when 3, Nokogiri::XML::Node::CDATA_SECTION_NODE
              stack.last[1] = reader.value

            #end element
            when 15
              return if stack.empty?

              elem = stack.pop
              parent = stack.last

              # I finished the node
              if parent.nil?
                yield(elem[1])
                elem = nil
                next
              end

              # else..
              key = elem[0]
              parent_childs = parent[1]
              if parent_childs.has_key?(key)
                unless parent_childs[key].is_a? Array
                  parent_childs[key] = [parent_childs[key]]
                end

                parent_childs[key] << elem[1]
              else
                parent_childs[key] =  elem[1]
              end
          end
        end
      end

References

ReXML

http://ruby-doc.org/core/classes/REXML/Parsers/SAX2Parser.html

http://stdlib.rubyonrails.org/libdoc/rexml/rdoc/classes/REXML/Parsers/PullParser.html

Nokogiri

http://nokogiri.rubyforge.org/nokogiri/Nokogiri/XML/SAX/Parser.html

http://nokogiri.rubyforge.org/nokogiri/Nokogiri/XML/Reader.html

LibXML

http://libxml.rubyforge.org/rdoc/

Posts on XML event parser

Comparing SAX parsers in Ruby

SO question on large XML

Transforming XML and the ReXML’s Pull Parser

Processing large XML files (SAX example)

Available on YAML or JSON format.

Data example:

- :airport_name: Charles de Gaulle International Airport (Roissy Airport)
  :complete_location: Paris, France
  :city: Paris
  :country: "France"
  :iata_code: CDG
  :icao_code: LFPG

Below is a snippet on how to parse this content using Hpricot.

        doc = Hpricot(open "http://en.wikipedia.org/wiki/List_of_airports_by_IATA_code:_#{letter}")

        doc.search('table.wikitable tr:not(:first-child):not(.sortbottom)').each do |tr_line|
          # Iata, Icao, Airport name, Location served
          line_records = (tr_line/'td').map{|el| ActionController::Base.helpers.strip_tags el.inner_html}
          next if line_records[2].blank?
          airport = {:iata_code         => line_records[0],
                     :icao_code         => line_records[1],
                     :airport_name      => line_records[2],
                     :complete_location => line_records[3],
          }

          location_array = airport[:complete_location].split(',')
          unless location_array.blank?
            airport[:city]    = location_array.first
            airport[:country] = location_array.last
          end

          result << airport
        end;
      end;

This post is about how this can be accomplished using sql views.

class CreateProducts < ActiveRecord::Migration

def self.up

sql = "

CREATE ALGORITHM = MERGE VIEW `products` (

id,

name,

class_name,

seller_type,

seller_id,

total_price,

created_at,

updated_at

) AS (

SELECT cars.id id, cars.name name,

'car' type,

cars.seller_type seller_type, cars.seller_id seller_id,

cars.total_price,

cars.created_at created_at, cars.updated_at updated_at

FROM cars

)

UNION (

SELECT bikes.id id, bikes.name name,

'bike' type,

bikes.seller_type seller_type, bikes.seller_id seller_id,

bikes.unit_price,

bikes.created_at created_at, bikes.updated_at updated_at

FROM bikes

)

ORDER BY updated_at

"

execute sql

end

def self.down

execute "DROP VIEW `products`"

end

end

class Product < ActiveRecord::Base
end

doc = Hpricot(open("http://en.wikipedia.org/wiki/List_of_country_calling_codes"))

table = (doc/"table")[4]
by_country = {}
table.search('/tr').each {|tr| tr0 = tr.search('/td')[0] ; tr1 = tr.search('/td')[1]; by_country[tr0.inner_html.gsub(/<.*>/,  "")] = tr1.search('/a').inner_html.gsub(" ", '').split(/\+/).select{|val| !val.empty?}.map(&:to_i) if tr1  } ; nil

Phone calling codes (MySQL dump)

Phone calling codes by country (YML format)

map = new GMap2(document.getElementById("map_canvas"));
geocoder = new GClientGeocoder();
function showAddress(address) {
  geocoder.getLocations(address, function (locations) {
  var north = locations.Placemark[0].ExtendedData.LatLonBox.north;
  var south = locations.Placemark[0].ExtendedData.LatLonBox.south;
  var east = locations.Placemark[0].ExtendedData.LatLonBox.east;
  var west = locations.Placemark[0].ExtendedData.LatLonBox.west;
  var bounds = new GLatLngBounds(new GLatLng(south, west),
  new GLatLng(north, east));
  var point = new GLatLng(locations.Placemark[0].Point.coordinates[1], locations.Placemark[0].Point.coordinates[0])
  map.setCenter(bounds.getCenter(), map.getBoundsZoomLevel(bounds));
  // Placing a marker in the center of the map
  var marker = new GMarker(point);
  map.addOverlay(marker);
});

It is a VERY BAD IDEEA to use floating point arithmetics to deal with currency. In most of the programming languages. Basically, because you’ll end up loosing money . And this (on the great majority of cases) is not desirable .

I’ll show you some magic (that you may try at home):
~$ ruby --version
ruby 1.8.7 (2009-06-12 patchlevel 174) [i486-linux]
~$ irb
irb(main):003:0> (10.12*100).to_i
=>; 1011

So, what just happened?

O.K. Now a little computer science lesson.

Floating Point (vs Fixed Point)

From wikipedia:

Numbers are in general represented approximately to a fixed number of significant digits and scaled using anexponent. The base for the scaling is normally 2, 10 or 16. The typical number that can be represented exactly is of the form:

significant digits × base^exponent

The term floating point refers to the fact that the radix point (decimal point, or, more commonly in computers, binary point) can “float”; that is, it can be placed anywhere relative to the significant digits of the number. This position is indicated separately in the internal representation, and floating-point representation can thus be thought of as a computer realization of scientific notation.

Short version:

- number is converted to scientific notation, and then coded in: sign bit, exponent, significand

- the precision is not fixed

- much larger range of representable numbers (much more than a fixed point representation), at the cost of precision

Decimal to Floating Point conversion

In this representation, a large group of rational numbers can not be represented in binary with a fixed number of digits.

For example, if you would like to convert 0.7 in binary format (the example is from http://sandbox.mc.edu/~bennet/cs110/flt/dtof.html) you’ll end up having an ‘endless’ number (in fact, a repeating fraction, in binary).

0.7	× 2 =	1.4	1	Generate 1 and continue with the rest.
0.4	× 2 =	0.8	0	Generate 0 and continue.
0.8	× 2 =	1.6	1	Generate 1 and continue with the rest.
0.6	× 2 =	1.2	1	Generate 1 and continue with the rest.
0.2	× 2 =	0.4	0	Generate 0 and continue.
0.4	× 2 =	0.8	0	Generate 0 and continue.
0.8	× 2 =	1.6	1	Generate 1 and continue with the rest.
0.6	× 2 =	1.2	1	Generate 1 and continue with the rest.
…

The reason why the process seems to continue endlessly is that it does. The number 7/10, which makes a perfectly reasonable decimal fraction, is a repeating fraction in binary, just as the faction 1/3 is a repeating fraction in decimal. (It repeats in binary as well.) We cannot represent this exactly as a floating point number. The closest we can come in four bits is .1011. Since we already have a leading 1, the best eight-bit number we can make is 1.1011.

A good simulator can be fount at http://babbage.cs.qc.edu/IEEE-754/Decimal.html

And now back to our issue..

Ruby floating point values are stored in binary format.
10.12 (decimal) can not be represented exactly in binary:

>> sprintf(“%0.50f”, 10.12)
=> “10.11999999999999921840299066388979554176330566406250″

The default conversion of float to string (e.g., for output) rounds
to 6 (or maybe 7) places and truncates trailing zeros:

>> sprintf(“%0.6f”, a).sub(/0+\Z/,”)
=> “10.12″

Float#to_i truncates:

>> sprintf(“%0.50f”, 10.12 * 100 )
=> “1011.99999999999988631316227838397026062011718750000000″
>> (10.12 * 100).to_i
=> 1011

A very usefull add/sub/mul/div can be found at http://www.ecs.umass.edu/ece/koren/arith/simulator/FPMul/ . A more detailed explanation from there:

A    + 1.0100001111010111000010100011110101110000101000111101 *23 = 10.12
B    + 1.1001000000000000000000000000000000000000000000000000 *26 = 100

A*B  + 1.1111100111111111111111111111111111111111111111111111|011 *29

Postnormalization Step

A*B  + 1.1111100111111111111111111111111111111111111111111111|01 *29

Round to Zero

A*B  + 1.1111100111111111111111111111111111111111111111111111 *29 = 1011.9999999999999

Further references

Wikipedia provides even more examples of accuracy problems:http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems.