Password encryption is not supported by the bundled jTDS driver so you must instead download and configure the jConnect driver with DbVisualizer first. Please see this page where to get the jConnect driver:

This Users Guide page describes how to load the JAR file into DbVisualizer:

Loading and Configuring Drivers Manually

Then download the Bouncy Castle Crypto APIs for Java, which is the following file:

Now open DbVisualizer and the Tools->Driver Manager and do as follows:

  1. Select the Sybase jConnect JDBC driver.
  2. In the right pane, make sure User Specified tab is selected,
  3. Click the open file button to the right in the tab,
  4. Locate the bcprov-jdk15on-155.jar you just downloaded and open it in the file chooser,
  5. Now the list of files will list both the driver jar file and the newly loaded bcprov-jdk15on-155.jar file,
  6. Close the driver manager window.

Back in the main window do as follows:

  1. In the main window select the database connection for Sybase,
  2. Open the Properties tab,
  3. Select the Driver Properties category in the list,
  4. Click the green plus icon,
  5. Enter property name JCE_PROVIDER_CLASS,
  6. Enter the property value org.bouncycastle.jce.provider.BouncyCastleProvider,
  7. Make also sure that the ENCRYPT_PASSWORD setting is set to true,
  8. Click Apply.

Now re-connect and test that it works.