machine learning : k means algoritm

[bluatigro]

error :
the point end up in one set

'' bluatigro 11 aug 2018
'' k means algoritm :
'' put points in random set
'' while point moved
''   comput centum of sets
''   move eatch point into closest set


global pointmax , setmax
pointmax = 9
setmax = 2

dim x( pointmax ) , y( pointmax ) , set( pointmax )
dim setx( setmax ) , sety( setmax )


for i = 0 to pointmax
 read a , b
 x( i ) = a
 y( i ) = b
 set( i ) = int( rnd(0) * 3 )
next i
data 73.0 , 72.6
data 61.0 , 54.4
data 67.0 , 99.9
data 68.0 , 97.3
data 62.0 , 59.0
data 75.0 , 81.6
data 74.0 , 77.1
data 66.0 , 97.3
data 68.0 , 93.3
data 61.0 , 59.0
WindowWidth = 600
WindowHeight = 600
nomainwin
open "k clustering ." for graphics as #m
 #m "trapclose [quit]"
 call showdata
 #m "when leftButtonDown [leftdown]"
 #m "setfocus"
wait
[leftdown]
 pointmove = 1
 while pointmove = 1
   pointmove = 0
   '' calc centrum of eatch set
   for i = 0 to pointmax
     setx( set( i ) ) = setx( set( i ) ) + x( i ) _
     / ( pointmax + 1 )
     sety( set( i ) ) = sety( set( i ) ) + y( i ) _
     / ( pointmax + 1 )
   next i
   '' see whitch setcentrum is closest to point
   for i = 0 to pointmax
     q = 10000
     q2 = -1
     for s = 0 to setmax
       q3 = dist( x(i) , y(i) , setx(s) , sety(s) )
       if q3 < q then
         q = q3
         q2 = s
       end if
     next s
     '' move point to closest set
     if q2 <> set(i) then
       pointmove = 1
       set(i) = q2
     end if
   next i
 wend
 call showdata
 notice "ready"
wait
function dist( x1 , y1 , x2 , y2 )
 dist = sqr( ( x1 - x2 ) ^ 2 + ( y1 - y2 ) ^ 2 )
end function
sub showdata
 #m "fill black"
 for i = 0 to pointmax
   #m "goto " ; x( i ) * 10 - 300 _
   ; " " ; y( i ) * 6 - 300
   select case set( i )
     case 0
       #m "backcolor red"
     case 1
       #m "backcolor green"
     case else
       #m "backcolor blue"
   end select
   #m "down"
   #m "circlefilled 10"
   #m "up"
 next i
end sub
[quit]
 close #m
end



[tenochtitlanuk]

I think the section that should work out the centrum from the average x and y of each set is wrong... there are not ( pointmax +1 ) points in each set.

[bluatigro]

update :
the centers are now calced whit eatchs set its pointnumber


error :
i see now 2 sets ad the end
it shoot be 3


'' bluatigro 13 aug 2018
'' k means algoritm :
'' put points in random set
'' while point moved
''   comput centum of sets
''   move eatch point into closest set


global pointmax , setmax
pointmax = 9
setmax = 2

dim x( pointmax ) , y( pointmax ) , set( pointmax )
dim setx( setmax ) , sety( setmax ) , set2( pointmax )
dim a( setmax ) , b( setmax )



for i = 0 to pointmax
 read a , b
 x( i ) = a
 y( i ) = b
 set( i ) = int( rnd(0) * ( setmax + 1 ) )
next i
data 73.0 , 72.6
data 61.0 , 54.4
data 67.0 , 99.9
data 68.0 , 97.3
data 62.0 , 59.0
data 75.0 , 81.6
data 74.0 , 77.1
data 66.0 , 97.3
data 68.0 , 93.3
data 61.0 , 59.0
WindowWidth = 600
WindowHeight = 600
nomainwin
open "k clustering ." for graphics as #m
 #m "trapclose [quit]"
 call showdata
 #m "when leftButtonDown [leftdown]"
 #m "setfocus"
wait
[leftdown]
 pointmove = 1
 while pointmove = 1
   pointmove = 0
   '' calc centrum of eatch set
   for i = 0 to setmax
     a( i ) = 0
     b( i ) = 0
   next i
   for i = 0 to pointmax
     setx( set( i ) ) = setx( set( i ) ) + x( i )
     a( set( i ) ) = a( set( i ) ) + 1
     sety( set( i ) ) = sety( set( i ) ) + y( i )
     b( set( i ) ) = b( set( i ) ) + 1
   next i
   for i = 0 to setmax
     setx( i ) = setx( i ) / ( a( i ) + 1 )
     sety( i ) = sety( i ) / ( b( i ) + 1 )
   next i
   '' see whitch setcentrum is closest to point
   for i = 0 to pointmax
     q = 10000
     q2 = -1
     for s = 0 to setmax
       q3 = dist( x(i) , y(i) , setx(s) , sety(s) )
       if q3 < q then
         q = q3
         q2 = s
       end if
     next s
     '' move point to closest set
     set2(i) = set(i)
     if q2 <> set(i) then
       pointmove = 1
       set2(i) = q2
     end if
   next i
   for i = 0 to pointmax
     set(i) = set2(i)
   next i
 wend
 call showdata
 notice "ready"
wait
function dist( x1 , y1 , x2 , y2 )
 dist = sqr( ( x1 - x2 ) ^ 2 + ( y1 - y2 ) ^ 2 )
end function
sub showdata
 #m "fill black"
 for i = 0 to pointmax
   #m "goto " ; x( i ) * 10 - 300 _
   ; " " ; y( i ) * 6 - 300
   select case set( i )
     case 0
       #m "backcolor red"
     case 1
       #m "backcolor green"
     case else
       #m "backcolor blue"
   end select
   #m "down"
   #m "circlefilled 10"
   #m "up"
 next i
end sub
[quit]
 close #m
end



[tenochtitlanuk]

Since you are now counting number in each set you don't need the +1 bit.

There's something else wrong after that section still.

I rewrote the code and my version works as expected. I drew on-screen circles around each centrum to see them move and settle. I think you do marvellously for someone with sight problems! Keep these interesting tasks coming....

[tenochtitlanuk]

For interest,

[bluatigro]

update :
i tryed it whit changed code

error :
the point's go to 1 set again

i realy wood like to see the good solution



'' bluatigro 15 aug 2018
'' k means algoritm :
'' put points in random set
'' while point moved
''   comput centum of sets
''   move eatch point into closest set


global pointmax , setmax
pointmax = 9
setmax = 2

dim x( pointmax ) , y( pointmax ) , set( pointmax )
dim setx( setmax ) , sety( setmax ) , set2( pointmax )
dim a( setmax ) , b( setmax )



for i = 0 to pointmax
 read a , b
 x( i ) = a
 y( i ) = b
 set( i ) = int( rnd(0) * ( setmax + 1 ) )
next i
data 73.0 , 72.6
data 61.0 , 54.4
data 67.0 , 99.9
data 68.0 , 97.3
data 62.0 , 59.0
data 75.0 , 81.6
data 74.0 , 77.1
data 66.0 , 97.3
data 68.0 , 93.3
data 61.0 , 59.0
WindowWidth = 600
WindowHeight = 600
nomainwin
open "k clustering ." for graphics as #m
 #m "trapclose [quit]"
 call showdata
 #m "when leftButtonDown [leftdown]"
 #m "setfocus"
wait
[leftdown]
 pointmove = 1
 while pointmove = 1
   pointmove = 0
   '' calc centrum of eatch set
   for i = 0 to setmax
     a( i ) = 0
     b( i ) = 0
   next i
   for i = 0 to pointmax
     setx( set( i ) ) = setx( set( i ) ) + x( i )
     a( set( i ) ) = a( set( i ) ) + 1
     sety( set( i ) ) = sety( set( i ) ) + y( i )
   next i
   for i = 0 to setmax
     if a( i ) > 0 then
       setx( i ) = setx( i ) / a( i )
       sety( i ) = sety( i ) / a( i )
     end if
   next i
   '' see whitch setcentrum is closest to point
   for i = 0 to pointmax
     q = 10000
     q2 = -1
     for s = 0 to setmax
       q3 = dist( x(i) , y(i) , setx(s) , sety(s) )
       if q3 < q then
         q = q3
         q2 = s
       end if
     next s
     '' move point to closest set
     set2(i) = set(i)
     if q2 <> set(i) then
       pointmove = 1
       set2(i) = q2
     end if
   next i
   for i = 0 to pointmax
     set(i) = set2(i)
   next i
 wend
 call showdata
 notice "ready"
wait
function dist( x1 , y1 , x2 , y2 )
 dist = sqr( ( x1 - x2 ) ^ 2 + ( y1 - y2 ) ^ 2 )
end function
sub showdata
 #m "fill black"
 for i = 0 to pointmax
   #m "goto " ; x( i ) * 10 - 300 _
   ; " " ; y( i ) * 6 - 300
   select case set( i )
     case 0
       #m "backcolor red"
     case 1
       #m "backcolor green"
     case else
       #m "backcolor blue"
   end select
   #m "down"
   #m "circlefilled 10"
   #m "up"
 next i
end sub
[quit]
 close #m
end




[tenochtitlanuk]

You were dividing by the number of occurrences to find the average BEFORE you had accumulated how many that was!

You can easily change it.

My version follows- still recognizably your code, but corrected and with contracting circles to show the stabilising of the three centrums.



'' bluatigro 15 aug 2018
'' k means algorithm :
'' put points in random set
'' while point moved
''   compute centrum of sets
''   move each point into closest set


'randomize 0.54321   '   so debugging always uses same randoms. REM out once OK.

global pointmax , setmax, turn
pointmax =   9
setmax   =   2
radius   = 100
turn     =   1

dim x( pointmax ) , y( pointmax ) , set( pointmax )
dim setx( setmax ) , sety( setmax ) , set2( pointmax )
dim a( setmax ) , b( setmax )

randomize 0.54321

for i = 0 to pointmax
 read a , b
 x( i ) = a
 y( i ) = b
 set( i ) = int( rnd(0) * ( setmax + 1 ) )'    allocate set at random
next i

data 73.0 , 72.6
data 61.0 , 54.4
data 67.0 , 99.9
data 68.0 , 97.3
data 62.0 , 59.0
data 75.0 , 81.6
data 74.0 , 77.1
data 66.0 , 97.3
data 68.0 , 93.3
data 61.0 , 59.0

WindowWidth  = 600
WindowHeight = 600

nomainwin

open "k clustering ." for graphics_nsb as #m

 #m "trapclose quit"
 call showdata
 #m "when leftButtonDown [leftdown]"
 #m "setfocus"

 wait

[leftdown]
 pointmove = 1
 now       = time$( "seconds")

 while ( pointmove = 1) or ( time$( "seconds") -now) < 10 '   add a 10 second time-out, in case enters a loop...
   scan
   pointmove = 0
   '' calc centroid of each set

   for i = 0 to setmax
     a( i )   = 0
     b( i )   = 0
     setx( i ) =0
     sety( i ) =0
   next i

   for i = 0 to pointmax
     setx( set( i ) ) = setx( set( i ) ) + x( i )  '   totals subset x's
     a(    set( i ) ) = a(    set( i ) ) + 1       '   counts subset number
     sety( set( i ) ) = sety( set( i ) ) + y( i )  '   totals subset y's
     b(    set( i ) ) = b(    set( i ) ) + 1       '   counts subset number. same as a().
   next i

'   setx and sety now contain current centrums
   for i = 0 to setmax
     setx( i) =setx( i) /a( i)
     sety( i) =sety( i) /b( i)
   next i

   #m "color white"
   for set =0 to setmax

       select case set
           case 0
               #m "color red"
           case 1
               #m "color green"
           case else
               #m "color blue"
       end select

       #m "up ; goto "; int( 10 *setx( set ) -300); " "; int( 6 *sety( set ) -300); " ; down ; circle "; radius
   next set

   radius =int( radius *0.7)

   #m "up"

   timer 2000, [o]
   wait
   [o]
   timer 0

   '' see which set's centroid is closest to point
   for i = 0 to pointmax
     q  = 10000
     q2 = -1

     for s = 0 to setmax
       q3 = dist( x(i) , y(i) , setx(s) , sety(s) )

       if q3 < q then
         q  = q3
         q2 = s
       end if

     next s

     '' move point to closest set
     set2( i ) = set( i )

     if q2 <> set( i ) then
       pointmove = 1
       set2(i)   = q2
     end if

   next i

   for i = 0 to pointmax
     set( i ) = set2( i )
   next i
 wend

 call showdata

 notice "Attention!" + chr$( 13 ) + "Ready"

 wait

 end

function dist( x1 , y1 , x2 , y2 )
 dist = sqr( ( x1 - x2 ) ^ 2 + ( y1 - y2 ) ^ 2 )
end function

sub showdata
 #m "fill black"

 for i = 0 to pointmax
   #m "goto " ; x( i ) * 10 - 300; " " ; y( i ) * 6 - 300

   if turn =1 then turn =2: #m "backcolor lightgray"
   #m "color black"
   if turn >1 then

       select case set( i )
           case 0
               #m "backcolor red"
           case 1
               #m "backcolor green"
           case 2
               #m "backcolor blue"
       end select
   end if

   #m "down"
   #m "circlefilled 10"
   #m "up"
 next i
end sub

sub quit k$
 timer 0
 close #m
 end
end sub


[cassey]

Hi. I'm interested in machine learning and I found that ax-dynamics.com offers this solution. But here I can see that you can write your own machine learning algorithm. I'm quite bad at coding, so I wanted to ask you about machine learning. Do i have to work with other company to get this solution or can I just do this myself like you?

[B+]

towardsdatascience.com/a-tour-of-the-top-10-algorithms-for-machine-learning-newbies-dde4edffae11

Looks like bluatigro is working on #6 Nearest Neighbor

[Rod]

Any time you spend with Just BASIC will not be wasted time. For a start it is free and uses very little resources to get going. If this is just for fun and for home use you will be able to code solutions. Will they run fast enough and handle enough data for commercial use? probably not.